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Abstract 

Randomization is an exceptional tool for the design of distributed algorithms, sometimes yield- 
ing efficient solutions to problems that are inherently complex, or even unsolvable, in the setting 
of deterministic algorithms. However, this tool has a price: even simple randomized algorithms 
can be extremely hard to verify and analyze. 

This thesis addresses the problem of verification of randomized distributed algorithms. We 
consider the problem both from the theoretical and the practical perspective. Our theoretical 
work builds a new mathematical model of randomized distributed computation; our practical 
work develops techniques to be used for the actual verification of randomized systems. Our 
analysis involves both untimed and timed systems, so that real-time properties can be investi- 
gated. 

Our model for randomized distributed computation is an extension of labeled transition 
systems. A probabilistic automaton is a state machine with transitions, where, unlike for labeled 
transition systems, a transition from a state leads to a discrete probability distribution over pairs 
consisting of a label and a state, rather than to a single label and a single state. A probabilistic 
automaton contains pure nondeterministic behavior since from each state there can be several 
transitions, and probabilistic behavior since once a transition is chosen the label that occurs and 
the state that is reached are determined by a probability distribution. The resolution of pure 
nondeterminism leads to probabilistic executions , which are Markov chain like structures. Once 
the pure nondeterminism is resolved, the probabilistic behavior of a probabilistic automaton 
can be studied. 

The properties of a randomized algorithm are stated in terms of satisfying some other prop- 
erty with a minimal or maximal probability no matter how the nondeterminism is resolved. 
In stating the properties of an algorithm we also account for the possibility of imposing re- 
strictions on the ways in which the nondeterminism is resolved (e.g., fair scheduling, oblivious 
scheduling,. . .). We develop techniques to prove the correctness of some property by reducing 
the problem to the verification of properties of non-randomized systems. One technique is 
based on coin lemmas, which state lower bounds on the probability that some chosen random 
draws give some chosen outcomes no matter how the nondeterminism is resolved. We identify 
a collection of progress statements which can be used to prove upper bounds to the expected 
running time of an algorithm. The methods are applied to prove that the randomized dining 
philosophers algorithm of Lehmann and Rabin guarantees progress in expected constant time 
and that the randomized algorithm for agreement of Ben- Or guarantees agreement in expected 
exponential time. 

To ensure that our new model has strong mathematical foundations, we extend some of the 



common semantics for labeled transition systems to the probabilistic framework. We define a 
compositional trace semantics where a trace is replaced by a probability distribution over traces, 
called a trace distribution, and we extend the classical bisimulation and simulation relations in 
both their strong and weak version. Furthermore, we define probabilistic forward simulations, 
where a state is related to a probability distribution over states. All the simulation relations 
are shown to be sound for the trace distribution semantics. 

In summary, we obtain a framework that accounts for the classical theoretical results of 
concurrent systems and that at the same time proves to be suitable for the actual verification 
of randomized distributed real-time systems. This double feature should lead eventually to the 
easy extension of several verification techniques that are currently available for non-randomized 
distributed systems, thus rendering the analysis of randomized systems easier and more reliable. 

Thesis Supervisor: Nancy A. Lynch 

Title: Professor of Computer Science 

Keywords: Automata, Distributed Algorithms, Formal Methods, Labeled Transition Systems, 

Randomized Systems, Real-Time Systems, Verification 
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Chapter 1 

Introduction 

1.1 The Challenge of Randomization 

In 1976 Rabin published a paper titled Probabilistic Algorithms [Rab76] where he presented 
efficient algorithms for two well-known problems: Nearest Neighbors, a problem in computa- 
tional geometry, and Primality Testing, the problem of determining whether a number is prime. 
The surprising aspect of Rabin's paper was that the algorithms for Nearest Neighbors and for 
Primality Testing were efficient, and the key insight was the use of randomized algorithms, 
i.e., algorithms that can flip fair coins. Rabin's paper was the beginning of a new trend of 
research aimed at using randomization to improve the complexity of existing algorithms. It is 
currently conjectured that there are no efficient deterministic algorithms for Nearest Neighbors 
and Primality Testing. 

Another considerable achievement came in 1982, when Rabin [Rab82] proposed a solution 
to a problem in distributed computing which was known to be unsolvable without random- 
ization. Specifically, Rabin proposed a randomized distributed algorithm for mutual exclusion 
between n processes that guarantees no-lockout (some process eventually gets to the critical 
region whenever some process tries to get to the critical region) and uses a test-and-set shared 
variable with O(logra) values. On the other hand, Burns, Fisher, Jackson, Lynch and Patter- 
son [BFJ + 82] showed that 0(n) values are necessary for a deterministic distributed algorithm. 
Since then, several other randomized distributed algorithms were proposed in the literature, 
each one breaking impossibility results proved for deterministic distributed algorithms. Several 
surveys of randomized algorithms are currently available; among those we cite [Kar90, GSB94]. 

The bottom line is that randomization has proved to be exceptionally useful for problems in 
distributed computation, and it is slowly making its way into practical applications. However, 
randomization in distributed computation leaves us with a challenge whose importance increases 
as the complexity of algorithms increases: 

LL How can we analyze randomized distributed algorithms? In particular, how can we 
convince ourselves that a randomized distributed algorithm works correctly?" 

The analysis of non-randomized distributed systems is challenging already, due to a phenomenon 
called nondeterminism. Specifically, whenever two systems run concurrently, the relative speeds 
of the two systems are not known in general, and thus it is not possible to establish a priori 
the order in which the systems complete their tasks. On the other hand, the ordering of the 
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completion of different tasks may be fundamental for the global correctness of a system, since, 
for example, a process that completes a task may prevent another process from completing 
its task. The structure of the possible evolutions of a system can become intricate quickly, 
justifying the statement "there is rather a large body of sad experience to indicate that a 
concurrent program can withstand very careful scrutiny without revealing its errors" [OL82]. 

The introduction of randomization makes the problem even more challenging since two 
kinds of nondeterminism arise. We call them pure nondeterminism and probabilistic nondeter- 
minism. Pure nondeterminism is the nondeterminism due to the relative speeds of different 
processes; probabilistic nondeterminism is the nondeterminism due to the result of some ran- 
dom draw. Alternatively, we refer to pure nondeterminism as the nondeterministic behavior of 
a system and to probabilistic nondeterminism as the probabilistic behavior of a system. The 
main difficulty with randomized distributed algorithms is that the interplay between probabil- 
ity and nondeterminism can create subtle and unexpected dependencies between probabilistic 
events; the experience with randomized distributed algorithms shows that "intuition often fails 
to grasp the full intricacy of the algorithm" [PZ86], and "proofs of correctness for probabilistic 
distributed systems are extremely slippery" [LR81]. 

In order to meet the challenge it is necessary to address two main problems. 

• Modeling: How do we represent a randomized distributed system? 

• Verification: Given the model, how do we verify the properties of a system? 

The main objective of this thesis is to make progress towards answering these two questions. 

1.1.1 Modeling 

First of all we need a collection of mathematical objects that describe a randomized algorithm 
and its behavior, i.e., we need a formal model for randomized distributed computation. The 
model needs to be sufficiently expressive to be able to describe the crucial aspects of randomized 
distributed computation. Since the interplay between probability and nondeterminism is one 
of the main sources of problems for the analysis of an algorithm, a first principle guiding our 
theory is the following: 

1. The model should distinguish clearly between probability and nondeterminism. 

That is, if either Alice or Bob is allowed to flip a coin, the choice of who is flipping a coin is 
nondeterministic, while the outcome of the coin flip is probabilistic. 

Since the model is to be used for the actual analysis of algorithms, the model should allow 
the description of randomized systems in a natural way. Thus, our second guiding principle is 
the following: 

2. The model should correspond to our natural intuition of a randomized system. 

That is, mathematical elegance is undoubtedly important, but since part of the verification 
process for an algorithm involves the representation of the algorithm itself within the formal 
model, the chance of making errors is reduced if the model corresponds closely to our view of 
a randomized algorithm. A reasonable tradeoff between theory and practice is necessary. 



14 



Our main intuition for a computer system, distributed or not, is as a state machine that 
computes by moving from one state to another state. This intuition leads to the idea of Labeled 
Transition Systems (LTS) [Kel76, PI08I]. A labeled transition system is a state machine with 
labels associated with the transitions (the moves from one state to another state). Labeled 
transition systems have been used successfully for the modeling of ordinary distributed systems 
[Mil89, Jon91, LV91, LT87, GSSL94], and for their verification [WLL88, SLL93, SGG+93, 
BPV94]; in this case the labels are used to model communication between several systems. Due 
to the wide use of labeled transition systems, the extensive collection of verification techniques 
available, and the way in which labeled transition systems correspond to our intuition of a 
distributed system, two other guiding principles for the thesis are the following: 

3. The new model should extend labeled transition systems. 

4. The extension of labeled transition systems should be conservative, i.e., whenever a system 
does not contain any random choices, our new system should reduce to an ordinary labeled 
transition system. 

In other words our model is an extension of the labeled transition system model so that ordinary 
non-randomized systems turn out to be a special case of randomized systems. Similarly, all the 
concepts that we define on randomized systems are generalizations of corresponding concepts 
of ordinary non-randomized systems. In this way all the techniques available should generalize 
easily without the need to develop completely new and independent techniques. Throughout 
the thesis we refer to labeled transition systems as automata and to their probabilistic extension 
as probabilistic automata. 

1.1.2 Verification 

Once the model is built, our primary goal is to use the model to describe the properties that 
a generic randomized algorithm should satisfy. If the model is well designed, the properties 
should be easy to state. Then, our second goal is to develop general techniques that can be 
used for verification. 

We investigate verification techniques from two perspectives. On one hand we formalize 
some of the kinds of the informal arguments that usually appear in existing papers; on the 
other hand we extend existing abstract verification techniques for labeled transition systems 
to the probabilistic framework. Examples of abstract techniques include the analysis of traces 
[Hoa85], which are ordered sequences of labels that can occur during the evolution of a system, 
and of simulation relations [Mil89, Jon91, LV91], which are relations between the states of 
two systems such that one system can simulate the transitions of the other via the simulation 
relation. To provide some intuition for traces and simulations, Figure 1-1 represents three 
labeled transition systems, denoted by Ai, A2, and A3. The empty sequence and the sequences 
a and ab are the traces of Ai, A2, and A3. For example, a computation that leads to ab is the 
one that starts from so, moves to si, and then to S3. The dotted lines from one state to another 
state (the arrows identify the from-to property) are examples of simulation relations from one 
automaton to the other. For example, consider the simulation relation from A3 to Ai- State sq 
of A3 is related to state sq of A2; states si and si of A3 are related to state si of A2; state S3 
of A3 is related to state S3 of Ai- The transition of A3 from sq to S2 with action a is simulated 
in A2 by the transition from sq to si with label a. There is a strong simulation also from A2 
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Figure 1-1: Simulation relations for automata. 

to A3 (each state s 8 - of Ai is related to state S{ of A3), from A\ to A2, and from A2 to A\. 
There is an even stronger relation between A\ and A2, which is called a bisimulation and is 
represented by the double- arrow dotted lines between the states of A\ and Ai- A bisimulation 
is an equivalence relation between the states of two automata. In this case each automaton can 
simulate the transitions of the other via the bisimulation relation. 

Direct Verification 

In the description of a randomized distributed algorithm pure nondeterminism represents the 
undetermined part of its behavior, namely, in what order the processes are scheduled. Schedul- 
ing processes is the activity of removing the nondeterminism, and the object that does the 
scheduling is usually referred to as a scheduler or an adversary. The intuition behind the name 
"adversary" is in proving the correctness of an algorithm a scheduler is viewed as a malicious 
entity that degrades the performance of the system as much as possible. 

Once the nondeterminism is removed, a system looks like a Markov chain, and thus it is 
possible to reason about probabilities. A common argument is then 

LL no matter how the scheduler acts, the probability that some good property holds is 
at least p." 

Actually, in most of the existing work p is 1, since the proofs are easier to carry out in this case. 
In this thesis we are interested in every p since we are concerned also with the time complexity 
of an algorithm. Throughout the thesis it will become clear why we need every p for the study 
of time complexity. 

One of our major goals is to remove from the informal arguments of correctness all "danger- 
ous" statements, i.e., all statements that rely solely on intuition rather than on actual deduc- 
tions, and yet keep the structure of a proof simple. In other words, we want to provide tools 
that allow people to argue as before with a significantly higher confidence that what they say is 
correct. Then, we want to develop techniques that allow us to decompose the verification task 
of complex properties into simpler verification tasks. This feature is important for scalability. 
Here we give examples of two issues that we believe to be important. 

• Make sure that you know what probability space you are working in. Or, at least, make 
sure that you are working in a probability space. This is a rule of thumb that is valid in 
other fields like Information Theory and Detection Theory. Probability is very tricky. The 
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fact that a specific probability space was not identified was the reason for a bug discovered 
by Saias [Sai92] in the original algorithm of Rabin [Rab82], later fixed by Kushilevitz and 
Rabin [KR92]. Of course, in order to make sure we know what probability spaces we are 
working in, we need some easy mechanisms to identify those probability spaces. Such 
mechanisms were not available in f982. 

• Avoid arguments of the kind "now the worst thing that can happen is the following." 
These arguments are usually based on the intuition that the designers have about their 
own algorithm. Specifically, as has happened in the past, the designers argue based on 
worst cases they can think of rather than the actual worst case. What is missing is a 
proof showing that the worst case has been identified. A much better statement would 
be "no matter what happens, something else will happen", since it does not require us to 
identify the worst scenario. Using our methodology, Aggarwal [Agg94] discovered a bug 
in an algorithm designed by himself and Kutten [AK93] which was due to an argument of 
the kind cited above. Similarly, we discovered a bug in the timing analysis of the mutual 
exclusion algorithm of Pnueli and Zuck [PZ86]. This bug arose for the same reason. 

The reader familiar with existing work, and in particular familiar with model checking, may 
be a bit puzzled at this point. There is a considerable amount of work on model checking 
of randomized distributed systems, and yet we are introducing new techniques. Furthermore, 
although there is some ongoing work on automating part of the proof methods developed in this 
thesis [PS95], we do not address any decidability issue here. Our favorite analogy to justify our 
approach is that we view model checking as the program "Mathematica", a popular program 
for symbolic manipulation of analytic expressions. If we are given a simple analytical problem, 
we can use Mathematica to get the solution from a computer. On the other hand, if we have 
a complex analytical problem, say a complex function that we have defined, and we want to 
verify that it respects some specific constraints, or maybe we want to find the constraints, then 
things are very different, since the problem in general is undecidable, i.e., not solvable by a 
computer. We can plot part of the given function using Mathematica and have a rough idea of 
whether it satisfies the desired constraints. If the plot shows that the function violates some 
of the constraints, then we have to change either the function or the constraints; if the plot 
shows that the function does not violate the constraints, then we can start to use all the tools 
of analysis to prove that the given function satisfies the constraints. In this way Mathematica 
saves us a lot of time. In using the analytical tools we need to use our creativity and our 
intuition about the problem so that we can solve its undecidable part. We view our research as 
building the analytical tools. 

Simulations 

The study of traces and simulations carried out in the thesis contributes more directly to theory 
than to practice. In particular, we do not give any examples of verification using simulations. 
However, due to the success that simulation relations have had for the verification of ordinary 
labeled transition systems, it is likely that the same methods will also work for randomized 
systems. 

A considerable amount of research has been carried out in extending trace semantics and 
simulation relations to the probabilistic case, especially within process algebras [Hoa85, Mil89, 
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BW90]; however, most of the existing literature does not address pure nondeterminism, and 
thus it has limited practical applicability. We believe it is important to have a model that is 
both useful for realistic problems and accounts for the existing theoretical work. In particu- 
lar, based on some of the interpretations that are given to nondeterminism within ordinary 
automata, we realize that, also in the probabilistic case, pure nondeterminism can be used to 
express much more than just the relative speeds of processes running concurrently. Specifically, 
nondeterminism can be used to model the following phenomena. 

1. Scheduling freedom. This is the classical use of nondeterminism, where several processes 
run in parallel and there is freedom in the choice of which process performs the next 
transition. 

2. External environment. Some of the labels can represent communication events due to the 
action of some external user, or more generally, to the action of an external environment. 
In this case nondeterminism models the arbitrary behavior of the external environment, 
which is chosen by an adversary. 

3. Implementation Freedom. A probabilistic automaton is viewed as a specification, and 
nondeterminism represents implementation freedom. That is, if from some state there 
are two transitions that can be chosen nondeterministically, then an implementation can 
have just one of the two transitions. In this case an adversary chooses the implementation 
that is used. 

It is important to recognize that, in the labeled transition system model, the three uses of 
nondeterminism described above can coexist within the same automaton. It is the specific 
interpretation that is given to the labels that determines what is expressed by nondeterminism 
at each point. 

1.2 Organization of the Thesis 

The thesis is divided in two main parts: the first part deals with the untimed model and the 
second part deals with the timed model. The second part relies heavily on the first part and 
adds a collection of results that are specific to the analysis of real-time properties. We describe 
the technical contributions of the thesis chapter by chapter. 

An Overview of Related Work. Chapter 2 gives an extensive overview of existing work 
on modeling and verification of randomized distributed systems. 

Preliminaries. Chapter 3 gives the basics of probability theory that are necessary to under- 
stand the thesis and gives an overview of the labeled transition systems model. All the topics 
covered are standard, but some of the notation is specific to this thesis. 

Probabilistic Automata. Chapter 4 presents the basic probabilistic model. A probabilistic 
automaton is a state machine whose transitions lead to a probability distribution over the labels 
that can occur and the new state that is reached. Thus, a transition describes the probabilistic 
behavior of a probabilistic automaton, while the choice of which transition to perform describes 
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the nondeterministic behavior of a probabilistic automaton. A computation of a probabilistic 
automaton, called a probabilistic execution, is the result of resolving the nondeterminism in a 
probabilistic automaton, i.e., the result of choosing a transition, possibly using randomization, 
from every point. A probabilistic execution is described essentially by an infinite tree with 
probabilities associated with its edges. On such a tree it is possible to define a probability 
space, which is the object through which the probabilistic properties of the computation can 
be studied. We extend the notions of finiteness, prefix and suffix of ordinary executions to 
the probabilistic framework and we extend the parallel composition operator. Finally, we show 
how to project a probabilistic execution of a compound probabilistic automaton onto one of 
its components and we show that the result is a probabilistic execution of the component. 
Essentially, we show that the properties of ordinary automata are preserved in the probabilistic 
framework. The probabilistic model is an extension of ordinary automata since an ordinary 
automaton can be viewed as a probabilistic automaton where each transition leads just to one 
action and one state. 

Direct Verification: Stating a Property. Chapter 5 shows how to formalize commonly 
used statements about randomized algorithms and shows how such formal statements can be 
manipulated. We start by formalizing the idea of an adversary, i.e., the entity that resolves 
the nondeterminism of a system in a malicious way. An adversary is a function that, given 
the past history of a system, chooses the next transition to be scheduled, possibly using ran- 
domization. The result of the interaction between an adversary and a probabilistic automaton 
is a probabilistic execution, on which it is possible to study probabilistic properties. Thus, 
given a collection of adversaries and a specific property, it is possible to establish a bound on 
the probability that the given property is satisfied under any of the given adversaries. We call 
such bound statements probabilistic statements . We show how probabilistic statements can be 
combined together to yield more complex statements, thus allowing for some form of compo- 
sitional verification. We introduce a special kind of probabilistic statement, called a progress 
statement, which is a probabilistic extension of the leads-to operator of UNITY [CM88]. Infor- 
mally, a progress statement says that if a system is started from some state in a set of states 
U , then, no matter what adversary is used, a state in some other set of states U' is reached 
with some minimum probability p. Progress statements can be combined together under some 
general conditions on the class of adversaries that can be used. 

Finally, we investigate the relationship between deterministic adversaries (i.e., adversaries 
that cannot use randomness in their choices) and general adversaries. We show that for a large 
class of collections of adversaries and for a large class of properties it is sufficient to analyze 
only deterministic adversaries in order to derive statements that concern general adversaries. 
This result is useful in simplifying the analysis of a randomized algorithm. 

Direct Verification: Proving a Property. Chapter 6 shows how to prove the validity 
of a probabilistic statement from scratch. We introduce a collection of coin lemmas, which 
capture a common informal argument on probabilistic algorithms. Specifically, for many proofs 
in the literature the intuition behind the correctness of an algorithm is based on the following 
fact: if some specific random draws give some specific results, then the algorithm guarantees 
success. Then, the problem is reduced to showing that, no matter what the adversary does, 
the specific random draws give the specific results with some minimum probability. The coin 
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lemmas can be used to show that the specific random draws satisfy the minimum probability 
requirement; then, the problem is reduced to verifying properties of a system that does not 
contain probability at all. Factoring out the probability from a problem helps considerably in 
removing errors due to unexpected dependencies. 

We illustrate the method by verifying the correctness of the randomized dining philosophers 
algorithm of Lehmann and Rabin [LR81] and the algorithm for randomized agreement with 
stopping faults of Ben-Or [B083]. In both cases the correctness proof is carried out by proving 
a collection of progress statements using some coin lemmas. 

Finally, we suggest another technique, called the partition technique, that departs consid- 
erably from the coin lemmas and that appears to be useful in some cases. We illustrate the 
partition technique on a toy resource allocation protocol, which is one of the guiding examples 
throughout Chapters 5 and 6. 

Hierarchical Verification: Trace Distributions. Chapter 7 extends the trace-based se- 
mantics of ordinary automata [Hoa85] to the probabilistic framework. A trace is a ordered 
sequence of labels that occur in an execution; a trace distribution is the probability distribu- 
tion on traces induced by a probabilistic execution. We extend the trace preorder of ordinary 
automata (inclusion of traces) to the probabilistic framework by defining the trace distribution 
preorder. However, the trace distribution preorder is not preserved by the parallel composition 
operator, i.e., it is not a precongruence. Thus, we define the trace distribution precongruence 
as the coarsest precongruence that is contained in the trace distribution preorder. Finally, we 
show that there is an elementary probabilistic automaton called the principal context that dis- 
tinguishes all the probabilistic automata that are not in the trace distribution precongruence 
relation. This leads us to an alternative characterization of the trace distribution precongruence 
as inclusion of principal trace distributions. 

Hierarchical Verification: Simulations. Chapter 8 extends the verification method based 
on simulation relations to the probabilistic framework. Informally, a simulation relation from 
one automaton to another automaton is a relation between the states of the two automata that 
allows us to embed the transition relation of one automaton in the other automaton. In the 
probabilistic framework a simulation relation is still a relation between states; however, since 
a transition leads to a probability distribution over states, in order to say that a simulation 
relation embeds the transition relation of a probabilistic automaton into another probabilistic 
automaton we need to extend a relation defined over states to a relation defined over probabil- 
ity distributions over states. We generalize the strong and weak bisimulation and simulation 
relations of Milner, Jonsson, Lynch and Vaandrager [Mil89, Jon91, LV91] to the probabilistic 
framework. Then, we introduce a coarser simulation relation, called a probabilistic forward 
simulation, where a state is related to a probability distribution over states rather than to a 
single state. We prove an execution correspondence theorem which, given a simulation relation 
from one probabilistic automaton to another probabilistic automaton, establishes a strong cor- 
respondence between each probabilistic execution of the first probabilistic automaton and one 
of the probabilistic executions of the second automaton. Based on the execution correspon- 
dence theorem, we show that each of the relations presented in the chapter is sound for the 
trace distribution precongruence. Thus, simulation relations can be used as a sound technique 
to prove principal trace distribution inclusion. 
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Probabilistic Timed Automata. Chapter 9 starts the second part of the thesis. We extend 
probabilistic automata with time following the approach of Lynch and Vaandrager [LV95], where 
passage of time is modeled by means of transitions labeled with positive real numbers. In order 
to use most of the untimed theory, we force time-passage transition not to be probabilistic. 
We extend probabilistic executions to the timed framework, leading to probabilistic timed 
executions, and we show the relationship between probabilistic executions and probabilistic 
timed executions. The main idea is that in several circumstances it is sufficient to analyze the 
probabilistic executions of a system in order to study its real-time behavior. 

Direct Verification: Time Complexity. Chapter 10 introduces new techniques for the 
verification of real-time properties of a randomized algorithm. The techniques of Chapter 5 
still apply; however, due to the presence of time, it is possible to study the time complexity 
of an algorithm. We augment the progress statements of Chapter 5 with an upper bound t to 
state the following: if a system is started from some state in a set of states U , then, no matter 
what adversary is used, a state of some other set of states U' is reached within time t with 
some minimum probability p. Based on these timed progress statements , we show how to derive 
upper bounds on the expected time to reach some set of states. We illustrate the technique 
by showing that the randomized dining philosophers algorithm of Lehmann and Rabin [LR81] 
guarantees progress within expected constant time. 

By extending the technique for the analysis of expected time, we show how to derive bounds 
on more abstract notions of complexity. In particular, we consider the algorithm for randomized 
agreement of Ben-Or as an example. The algorithm of Ben-Or runs in stages. From the way 
the algorithm is structured, it is not possible to give meaningful bounds on the time it takes 
to make progress from any reachable state. However, using abstract complexities, it is easy 
to prove an upper bound on the expected number of stages that are necessary before reaching 
agreement. Once an upper bound on the expected number of stages is derived, it is easy to 
derive an upper bound on the expected time to reach agreement. 

Hierarchical Verification: Timed Trace Distributions and Timed Simulations. Chap- 
ters 11 and 12 extend the trace distribution precongruence and the simulation relations of the 
untimed framework to the timed framework. A trace is replaced by a timed trace, where a 
timed trace is a sequence of labels paired with their time of occurrence plus a limit time. The 
timed trace distribution precongruence is characterized by a timed principal context, which is 
the principal context augmented with arbitrary time-passage transitions. All the timed simu- 
lation relations are shown to be sound for the timed trace distribution precongruence. All the 
results are proved by reducing the problem to the untimed framework. 

Conclusion. Chapter 13 gives some concluding remarks and several suggestions for further 
work. Although this thesis builds a model for randomized computation and shows that it is 
sufficiently powerful for the analysis of randomized distributed real-time algorithms, it just 
discovers the tip of the iceberg. We propose a methodology for the analysis of randomization, 
and we give several examples of the application of such methodology; however, there are several 
other ways to apply our methodology. It is very likely that new probabilistic statements, new 
results to combine probabilistic statements, and new coin lemmas can be developed based on the 
study of other algorithms; similarly, the fundamental idea behind the trace semantics that we 
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present can be used also for other kinds of observational semantics like failures [Hoa85, DH84]. 
We give hints on how it is possible to handle liveness within our model and state what we know 
already. Furthermore, we give ideas of what is possible within restricted models where some 
form of I/O distinction like in the work of Lynch and Tuttle [LT87] or some timing restriction 
like in the work of Merritt, Modugno and Tuttle [MMT91] is imposed. Finally, we address the 
issue of relaxing some of the restrictions that we impose on the timed model. 

1.3 Reading the Thesis 

The two parts of the thesis, the untimed and the timed part, proceed in parallel: each chapter of 
the untimed part is a prerequisite for the corresponding chapter in the timed part. Each part is 
subdivided further into two parts: the direct verification and the hierarchical verification. The 
two parts can be read almost independently, although some knowledge of the direct verification 
method can be of help in reading the hierarchical method. The direct method is focused mainly 
on verification of algorithms, while the hierarchical method is focused mainly on the theoretical 
aspects of the problem. Further research should show how the hierarchical method can be of 
significant help for the analysis of randomized algorithms. 

Each chapter starts with an introductory section that gives the main motivations and an 
overview of the content of the chapter. Usually, the more technical discussion is concentrated 
at the end. The same structure is used for each section: the main result and short proofs are 
at the beginning of each section, while the long proofs and the more technical details are given 
at the end. A reader can skip the proofs and the most technical details on a first reading in 
order to have a better global picture. It is also possible to read just Chapter 3 and the first 
section (including subsections) of Chapters 4 to 12, and have a global view of the results of 
the thesis. In a second reading, the interested reader can concentrate on the proofs and on the 
technical definitions that are necessary for the proofs. The reader should keep in mind that 
several proofs in the thesis are based on similar techniques. Such techniques are explained in 
full detail only the first time they are used. 

A reader interested only in the techniques for the direct verification of algorithms and not 
interested in the arguments that show the foundations of the model can avoid reading the proofs. 
Moreover, such a reader can just glance over Section 4.2.6, and skip Sections 4.2.7, 4.3, and 4.4. 
In the timed framework the reader interested just in the techniques for the direct verification 
of algorithms can skip all the comparison between the different types of probabilistic timed 
executions and concentrate more on the intuition behind the definition of a probabilistic timed 
execution. 
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Chapter 2 

An Overview of Related Work 



In this chapter we give an extensive overview of existing work on modeling and verification of 
randomized distributed systems. We defer the comparison of our work with the existing work 
to the end of each chapter. Some of the descriptions include technical terminology which may 
be difficult to understand for a reader not familiar with concurrency theory. Such a reader 
should focus mainly on the high level ideas and not worry about the technical details. The rest 
of the thesis presents our research without assuming any knowledge of concurrency theory. We 
advise the reader not familiar with concurrency theory to read this chapter again after reading 
the thesis. 

There have been two main research directions in the field of randomized distributed real-time 
systems: one focused mainly on modeling issues using process algebras [Hoa85, Mil89, BW90] 
and labeled transition systems [Kel76, PI08I] as the basic mathematical objects; the other 
focused mainly on verification using Markov chains as the basic model and temporal logic 
arguments [Pnu82] and model checking [EC82, CES83] as the basic verification technique. Most 
of the results of the first of the research directions fail to model pure nondeterminism, while 
the results of the second of the research directions model pure nondeterminism successfully, but 
not in its full generality. As expressed at the end of Section 1. 1. 2, pure nondeterminism arises 
only in the choice of what process is performing the next instruction at each moment. Below 
we summarize the results achieved in both of the research directions. Furthermore, at the end 
of each chapter we add a section where we explain how the results described in this section are 
related to our research. 

2.1 Reactive, Generative and Stratified Models 

We present some of the existing work on modeling which is based on a classification due to van 
Glabbeek, Smolka, Steffen and Tofts [GSST90]. They define three types of processes: reactive, 
generative, and stratified. 

• Reactive model: Reactive processes consist of states and labeled transitions associated 
with probabilities. The restriction imposed on a reactive process is that for each state the 
sum of the probabilities of the transitions with the same label is I. 

• Generative model: Generative processes consist of states and labeled transitions associated 
with probabilities. The restriction imposed on a generative process is that for each state 
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Figure 2-1: Reactive, generative and stratified processes, from left to right. 

either there are no outgoing transitions, or the sum of the probabilities of all the outgoing 
transitions is 1. 

• Stratified model: Stratified processes consist of states, unlabeled transitions associated 
with probabilities, and labeled transitions. The restriction imposed on a stratified process 
is that for each state either there is exactly one outgoing labeled transition, or all the 
outgoing transitions are unlabeled and the sum of their probabilities is 1. 

Figure 2-1 gives an example of a reactive, a generative, and a stratified process. Informally, 
reactive processes specify for each label (also called action) the probability of reaching other 
states; generative processes also give additional information concerning the relative probabili- 
ties of the different actions; stratified processes add some probabilistic structure to generative 
processes. Observe that among the three models above only the reactive model has a struc- 
ture that can be used to express some form of pure nondeterminism (what action to perform), 
although in van Glabbeek et al. [GSST90] this issue is not considered. 

2.1.1 Reactive Model 

Rabin [Rab63] studies the theory of probabilistic automata, which are an instance of the reactive 
model. He defines a notion of a language accepted by a probabilistic automaton relative to a 
cut point A and shows that there are finite state probabilistic automata that define non-regular 
languages. 

Larsen and Skou [LS89, LS91] define a bisimulation type semantics, called probabilistic 
bisimulation, and a logic, called probabilistic model logic (PML), for reactive processes, and 
they introduce a notion of testing based on sequential tests and a copying facility. They show 
that two processes that satisfy the minimal probability assumption are probabilistically bisim- 
ilar if and only if they satisfy exactly the same PML formulas, and that two processes that 
satisfy the minimal probability assumption and that are not probabilistically bisimilar can be 
distinguished through testing with a probability arbitrarily close to 1. The minimum proba- 
bility assumption states that for every state the probability of each transition is either or is 
above some minimal value. This condition corresponds to the image-finiteness condition for 
non-probabilistic processes. Bloom and Meyer [BM89] relate the notions of probabilistic and 
non-probabilistic bisimilarity by showing that two non-probabilistic finitely branching processes 
P and Q are bisimilar if and only if there exists an assignment of probabilities to the transi- 
tions of P and Q such that the corresponding reactive processes P' and Q' are probabilistically 
bisimilar. 

Larsen and Skou [LS92] introduce a synchronous calculus for reactive processes where the 
probabilistic behavior is obtained through a binary choice operator parameterized by a prob- 
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ability p. They define a bisimulation relation on the new calculus, and they introduce a new 
extended probabilistic logic (EPL) which extends PML in order to support decomposition with 
respect to parallel composition. Both the probabilistic bisimulation and the extended proba- 
bilistic logic are axiomatized. 

2.1.2 Generative and Stratified Models 

Giacalone, Jou and Smolka [GJS90] define a process algebra for generative processes, called 
PCCS, which can be seen as a probabilistic extension of Milner's SCCS [Mil93]. In PCCS two 
processes synchronize at every transition regardless of the action that they perform. That is, if 
one process performs a transition labeled with action a with probability p a and another process 
performs a transition labeled with b with probability p&, then the two processes together can 
perform a transition labeled with ab with probability p a Pb- The authors provide an equational 
theory for PCCS based on the probabilistic bisimulation of Larsen and Skou [LS89], and provide 
an axiomatization for probabilistic bisimulation (the axiomatization is shown to be sound and 
complete in [JS90]). Furthermore, the authors define a notion of e-bisimulation, where two 
processes can simulate each other's transition with a probability difference at most e. Based on 
e-bisimulation, the authors define a metric on generative processes. 

Jou and Smolka [JS90] define trace and failure equivalence for generative processes. They 
show that, unlike for nondeterministic transition systems, maximality of traces and failures does 
not increase the distinguishing power of trace and failure equivalence, where by maximality of 
a trace we mean the probability to produce a specific trace and then terminate. More precisely, 
knowing the probability of each finite trace of a generative process gives enough information to 
determine the probability that a finite trace occurs leading to termination; similarly, knowing 
the probability of every failure of a generative process gives enough information to determine 
the probability of each maximal failure. Jou and Smolka show also that the trace and failure 
equivalences are not congruences. Our probabilistic executions are essentially generative pro- 
ceses, and our trace distributions are essentially the trace semantics of Jou and Smolka. In our 
case the properties shown by Jou and Smolka follow directly from measure theory. 

Van Glabbeek et al. [GSST90] state that the generative model is more general than the 
reactive model in the sense that generative processes, in addition to the relative probabilities 
of transitions with the same label, contain information about the relative probabilities of tran- 
sitions with different labels. They show also that the stratified model is a generalization of the 
generative model in the sense that a probabilistic choice in the generative model is refined by 
a structure of probabilistic choices in the stratified model. Formally, the authors give three 
operational semantics to PCCS, one reactive, one generative, and one stratified, and show how 
to project a stratified process into a generative process and how to project a generative process 
into a reactive process, so that the operational semantics of PCCS commute with the projec- 
tions. The reactive and generative processes of Figure 2-1 are the result of the projection of 
the generative and stratified processes, respectively, of Figure 2-1. Finally, the authors define 
probabilistic bisimulation for the generative and for the stratified models and show that bisim- 
ulation is a congruence in all the models and that bisimulation is preserved under projection 
from one model to the other. The results of van Glabbeek et al. [GSST90], however, are based 
on the fact that parallel composition is synchronous. 

Tofts [Tof90] introduces a weighted synchronous calculus whose operational semantics resem- 
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bles the stratified model. The main difference is that the weights associated with the transitions 
are not probabilities, but rather frequencies , and thus their sums are not required to be 1. Tofts 
defines two bisimulation relations that are shown to be congruences. The first relation is sensi- 
tive to the actual frequencies of the transitions leaving from a state, while the second relation 
is sensitive only to the relative frequencies of the transitions leaving from a state. In particular, 
the second relation coincides with the stratified bisimulation of van Glabbeek et al. [GSST90] 
after normalizing to 1 the frequencies of the transitions that leave from every state. The ad- 
vantage of Tofts' calculus is that it is not necessary to restrict the syntax of the expressions so 
that the weights of the choices at any point sum to 1 (such a restriction is imposed in PCCS). 
Moreover, it is possible to define a special weight uj that expresses infinite frequency and can 
be used to express priorities. A similar idea to express priorities is used by Smolka and Steffen 
in [SS90], where the stratified semantics of PCCS is extended with O-probability transitions. 

Baeten, Bergstra and Smolka [BBS92] define an algebra, prACFj , which is an extension 
of ACP [BW90] with generative probabilities. The authors show that prACP^ and a weaker 
version of ACP (ACPj ) are correlated in the sense that ACPj is the homomorphic image 
of prACFj in which the probabilities are forgotten. The authors also provide a sound and 
complete axiomatization of probabilistic bisimulation. 

Wu, Smolka and Stark [WSS94] augment the I/O automaton model of Lynch and Tuttle 
[LT87] with probability and they study a compositional behavioral semantics which is also 
shown to be fully abstract with respect to probabilistic testing. A test is a probabilistic I/O 
automaton with a success action w. The model is reactive for the input actions and generative 
for the output actions. This allows the authors to define a meaningful parallel composition 
operator, where two probabilistic I/O automata synchronize on their common actions and 
evolve independently on the others. In order to deal with the nondeterminism that arises from 
parallel composition, the authors attach a delay parameter to each state of a probabilistic I/O 
automaton, which can be seen as the parameter of an exponential probability distribution on 
the time of occurrence of the next local (i.e., output or internal) action. Whenever there is a 
conflict for the occurrence of two local actions of different probabilistic I/O automata, the delay 
parameters associated with the states are used to determine the probability with which each 
action occurs. The behavior of a probabilistic I/O automaton A is a function £ that associates 
a functional Eg with each finite trace fi . If the length of fi is n, then Eg takes a function / 
that given n + 1 delay parameters computes an actual delay, and returns the expected value of 
/ applied to the delay parameters of the computations of A that lead to ft. 

2.2 Models based on Testing 

Research on modeling has also focused on extending the testing preorders of De Nicola and 
Hennessy [DH84] to probabilistic processes. To define a testing preorder it is necessary to 
define a notion of a test and of how a test interacts with a process. The interaction between 
a test and a process may lead to success or failure. Then, based on the success or failure of 
the interactions between a process and a test, a preorder relation between processes is defined. 
Informally, a test checks whether a process has some specific features: if the interaction between 
a test and a process is successful, then the process has the desired feature. 

Ivan Christoff [Chr90b, Chr90a] analyzes generative processes by means of testing. A test 
is a nondeterministic finite-state process, and the interaction between a process and a test is 
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obtained by performing only those actions that both the processes offer and by keeping the 
relative probability of each transition unchanged. Four testing preorders are defined, each one 
based on the probability of the traces of the interaction between a process and a test. Christoff 
also provides a fully abstract denotational semantics for each one of the testing preorders: each 
process is denoted by a mapping that given an offering and a trace returns a probability. An 
offering is a finite sequence of non-empty sets of actions, and, informally, describes the actions 
that the environment offers to a process during the interaction between the process and a test. 

Linda Christoff [Chr93] builds on the work of Ivan Christoff and defines three linear se- 
mantics for generative processes: the trace semantics, the broom semantics, and the barbed 
semantics. The relations are defined in a style similar to the denotational models of Ivan 
Christoff, and, in particular, the trace and barbed semantics coincide with two of the semantics 
of [Chr90b]. Linda Christoff also defines three linear-time temporal logics that characterize her 
three semantics and provides efficient model checking algorithms for the recursion-free version 
of the logics. 

Testing preorders that are more in the style of De Nicola and Hennessy [DH84] are presented 
by Yi and Larsen in [YL92], where they define a process algebra with all the operators of CCS 
plus a binary probabilistic choice operator parameterized by a probability p. Thus, the calculus 
of Yi and Larsen allows for nondeterminism. A test is a process of their calculus with an 
additional label w. Depending on how the nondeterminism is resolved, w occurs with different 
probabilities in the interaction between a process and a test. Then, Yi and Larsen define a may 
preorder, which is based on the highest probability of occurrence of w, and a must preorder, 
which is based on the lowest probability of occurrence of w. The two preorders are shown to 
coincide with the testing preorders of De Nicola and Hennessy [DH84] when no probability is 
present. In more recent work Jonsson, Ho-Stuart and Yi [JHY94] give a characterization of 
the may preorder based on tests that are not probabilistic, while Jonsson and Yi [JY95] give a 
characterization of the may and must preorders based on general tests. 

Cleaveland, Smolka and Zwarico [CSZ92] introduce a testing preorder on reactive processes. 
A test is a reactive process with a collection of successful states and a non-observable action. 
The interaction between a test and a process allows an observable action to occur only if 
the two processes allow it to occur, and allows the non-observable action to occur if the test 
allows it to occur. The result is a generative process, where each of the actions that occur is 
chosen according to a uniform distribution (thus the formalism works only for finitely many 
actions). Two processes are compared based on the probability of reaching a successful state in 
the interaction between a process and a test. The authors show that their testing preorder is 
closely connected to the testing preorders of De Nicola and Hennessy [DH84] in the sense that 
if a process passes a test with some non-zero probability, then the non-probabilistic version 
of the process (the result of removing the probabilities from the transition relation of the 
process) may pass the non-probabilistic version of the test, and if a process passes a test with 
probability 1, then the non-probabilistic version of the process must pass the non-probabilistic 
version of the test. An alternative characterization of the testing preorder of Cleaveland et al. 
[CSZ92] is provided by Yuen, Cleaveland, Dayar and Smolka [YCDS94]. A process is represented 
as a mapping from probabilistic traces to [0,1], where a probabilistic trace is an alternating 
sequence of actions and probability distributions over actions. Yuen et al. use the alternative 
characterization to show that the testing preorder of Cleaveland et al. [CSZ92] is an equivalence 
relation. 
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2.3 Models with Nondeterminism and Denotational Models 

2.3.1 Transitions with Sets of Probabilities 

Jonsson and Larsen [JL91] introduce a new kind of probabilistic transition system where the 
transitions are labeled by sets of allowed probabilities. The idea is to model specifications where 
the probabilities associated with the transitions are not completely specified. They extend the 
bisimulation of Larsen and Skou [LS89] to the new framework and they propose two criteria for 
refinement between specifications. One criterion is analogous to the definition of simulations 
between non-probabilistic processes; the other criterion is weaker and regards a specification 
as a set of probabilistic processes. Refinement is then defined as inclusion of probabilistic 
processes. Finally, Jonsson and Larsen present a complete method for verifying containment 
between specifications. 

2.3.2 Alternating Models 

Hansson and Jonsson [HJ89, HJ90] develop a probabilistic process algebra based on an alternat- 
ing model. The model of Hansson and Jonsson, which is derived from the Concurrent Markov 
Chains of Vardi [Var85], is a model in which there are two kinds of states: probabilistic states, 
whose outgoing transitions are unlabeled and lead to nondeterministic states, and nondetermin- 
istic states, whose outgoing transitions are labeled and lead to probabilistic states. Only the 
transitions leaving from probabilistic states are probabilistic, and for each probabilistic state 
the probabilities of the outgoing transitions add to 1. The authors define a strong bisimulation 
semantics in the style of Larsen and Skou [LS89] for which they provide a sound and complete 
axiomatization. The model of Hansson and Jonsson [HJ90] differs substantially from the models 
of van Glabbeek et al. [GSST90] in that there is a clear distinction between pure nondeterminism 
and probability. The model could be viewed as an instance of the reactive model; however, the 
parallel composition operation defined by Hansson and Jonsson [HJ90] is asynchronous, while 
the classification of van Glabbeek et al. [GSST90] works only for synchronous composition. A 
complete presentation of the work of Hansson and Jonsson [HJ89, HJ90] appears in Hansson's 
PhD thesis [Han91], later published as a book [Han94]. Our simple probabilistic automata are 
very similar in style to the objects of Hansson's book. 

2.3.3 Denotational Semantics 

Seidel [Sei92] extends CSP [Hoa85] with probability. The extension is carried out in two steps. 
In the first step a process is a probability distribution over traces; in the second step, in order 
to account for the nondeterministic behavior of the environment, a process is a conditional 
probability measure, i.e., an object that given a trace, which is meant to be produced by the 
external environment, returns a probability distribution over traces. 

Jones and Plotkin [JP89] use a category theoretic approach to define a probabilistic pow- 
erdomain, and they use it to give a semantics to a language with probabilistic concurrency. 
It is not known yet how the semantics of Jones and Plotkin compares to existing operational 
semantics. 
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2.4 Models with Real Time 

There are basically two models that address real time issues. One model is the model of Hansson 
and Jonsson [Han94], where special \ actions can appear in the transitions. The occurrence of 
an action \ means that time has elapsed, and the amount of time that elapses in a computation 
is given by the number of occurrences of action \- Thus, the time domain of Hansson and 
Jonsson's model is discrete. 

The other model is based on stochastic process algebras and is used in the field of performance 
analysis. In particular, actions are associated with durations, and the durations are expressed 
by random variables. In order to simplify the analysis, the random variables are assumed to have 
an exponential probability distribution, which is memoryless. Research in this area includes 
work from Gotz, Herzog and Rettelbach [GHR93], from Hillston [Hil94], and from Bernardo, 
Donatiello and Gorrieri [BDG94]. 

2.5 Verification: Qualitative and Quantitative Methods 

Most of the research on the verification of randomized distributed systems is concerned with 
properties that hold with probability 1. The advantage of such properties is that for finite 
state processes they do not depend on the actual probabilities of the transitions, but rather on 
whether those transitions have probability or probability different from 0. Thus, the problem 
of checking whether a system satisfies a property with probability 1 is reduced to the problem 
of checking whether a non-randomized system satisfies some other property. This method is 
called qualitative, as opposed to the quantitative method, where probabilities different from 1 
also matter. 

The rationale behind the qualitative method is that a randomized process, rather than 
always guaranteeing success, usually guarantees success with probability 1, which is practically 
the same as guaranteeing success always. The quantitative method becomes relevant whenever 
a system has infinitely many states or the complexity of an algorithm needs to be studied. 

Almost all the papers that we describe in this section are based on a model where n Markov 
chains evolve concurrently. Each Markov chain represents a process, and the pure nondeter- 
minism arises from the choice of what Markov chain performs the next transition (what process 
is scheduled next). The object that resolves the nondeterminism is called a scheduler or adver- 
sary, and the result of a scheduler on a collection of concurrent Markov chains is a new Markov 
chain that describes one of the possible evolutions of the global system. Usually a scheduler is 
required to be fair in the sense that each process should be scheduled infinitely many times. 

2.5.1 Qualitative Method: Proof Techniques 

Huart, Sharir and Pnueli [HSP83] consider n finite state asynchronous randomized processes 
that run in parallel, and provide two necessary and sufficient conditions to guarantee that a 
given set of goal states is reached with probability 1 under any fair scheduler. A scheduler is 
the entity that at any point chooses the next process that performs a transition. The result 
of the action of a scheduler on n processes is a Markov chain, on which it is possible to study 
probabilities. A scheduler is fair if and only if, for each path in the corresponding Markov 
chain, each process is scheduled infinitely many times. The authors show that in their model 
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each property described by reaching a collection of states has either probability or probability 
1. Then, they describe a decision procedure for the almost sure reachability of a set of goal 
states. The procedure either constructs a decomposition of the state space into a sequence of 
components with the property that any fair execution of the program must move down the 
sequence with probability 1 until it reaches the goal states (goal states reached with probability 
1), or finds an ergodic set of states through which the program can loop forever with probability 
1 (goal states reached with probability 0). Finally the authors give some examples of problems 
where the use of randomization does not provide any extra power over pure nondeterminism. 
The proof principle of [HSP83] is generalized to the infinite state case by Hart and Sharir 
[HS85]. 

Lehmann and Shelah [LS82] extend the temporal logic of linear time of Pnueli [Pnu82] to 
account for properties that hold with probability 1, and they provide three complete axioma- 
tizations of the logic: one axiomatization is for general models, one is for finite models, and 
one is for models with bounded transition probabilities (same as the minimum probability re- 
quirement of Larsen and Skou [LS91]). A model of the logic is essentially a Markov chain, 
or alternatively an unlabeled generative process. The logic of Lehmann and Shelah [LS82] is 
obtained from the logic of Pnueli [Pnu82] by adding a new modal operator V whose meaning 
is that the argument formula is satisfied with probability 1. 

Pnueli [Pnu83] introduces the notion of extreme fairness and shows that a property that 
holds for all extreme fair executions holds with probability 1. Furthermore, Pnueli presents a 
sound proof rule based on extreme fairness and linear temporal logic. The model consists of n 
randomized processes in parallel. Each process is a state machine where each state enables a 
probabilistic transition, which lead to several modes. Resolving the nondeterminism leads to a 
Markov chain. However, only those Markov chains that originate from fair scheduling policies 
are considered. Then, an execution (a path in the Markov chain) is extremely fair relative 
to a property (f> (<f) is a property that is satisfied by states) if and only if for each transition 
that occurs infinitely many times from states that satisfy (f>, each mode of the transition occurs 
infinitely many times. An execution is extremely fair if and only if it is extremely fair relative 
to any formula (f> expressed in the logic used in [Pnu83]. The proof rule of Pnueli [Pnu83], 
along with some other new rules, is used by Pnueli and Zuck [PZ86] to verify two non-trivial 
randomized algorithms, including the Randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81]. Zuck [Zuc86] introduces the notion of a-fairness and shows that a-fairness 
is complete for temporal logic properties that hold with probability 1. 

Rao [Rao90] extends UNITY [CM88] to account for randomized systems and properties 
that hold with probability 1. The main emphasis is on properties rather than states. A new 
notion of weak probabilistic precondition is introduced that, together with the extreme fairness 
of Pnueli, generalizes weakest preconditions. Finally, based on the work of Huart et al. [HSP83], 
Rao argues that his new logic is complete for finite state programs. 

2.5.2 Qualitative Method: Model Checking 

Vardi [Var85] presents a method for deciding whether a probabilistic concurrent finite state 
program satisfies a linear temporal logic specification, where satisfaction means that a formula 
is satisfied with probability 1 whenever the scheduler is fair. A program is given as a Concurrent 
Markov Chain, which is a transition system with nondeterministic and probabilistic states. A 
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subset F of the nondeterministic states is called the set of fair states. A scheduler is a function 
that, based on the past history of a program, chooses the next transition to perform from 
a nondeterministic state. The result of the action of a scheduler on a program is a Markov 
chain on which it is possible to study the probability that some linear temporal logic formula 
is satisfied. A path in the Markov chain is fair if for each fair state that occurs infinitely many 
times each one of the possible nondeterministic choices from that state occurs infinitely many 
times; a scheduler is fair if the fair paths have probability 1 in the corresponding Markov chain. 
The model checking algorithm of Vardi works in time polynomial in the size of the program and 
doubly exponential in the size of the specification. By considering a slightly restricted logic, 
Vardi and Wolper [VW86] reduce the complexity of the model checking algorithm to only one 
exponent in the size of the formula. 

Courcoubetis and Yannakakis [CY88, CY90] investigate the complexity of model checking 
linear time propositional temporal logic of sequential and concurrent probabilistic processes. A 
sequential process is a Markov chain and a concurrent process is a Concurrent Markov Chain. 
They give a model checking algorithm that runs in time linear in the size of the program and 
exponential in the size of the formula, and they show that the problem is in PSPACE. Moreover, 
they give an algorithm for computing the exact probability with which a sequential program 
satisfies a formula. 

Alur, Courcoubetis and Dill [ACD91a, ACD91b] develop a model checking algorithm for 
probabilistic real-time systems. Processes are modeled as a generalized semi-Markov process, 
which are studied in [Whi80, She87]. Essentially a process is a finite state transition system 
with timing constraints expressed by probability distributions on the delays. They impose the 
restriction that every distribution is either discrete, or exponential, or has a density function 
which is different from only on a finite collection of intervals (in [ACD91a] only this last case 
is studied). The temporal logic, called TCTL, is an extension of the branching-time temporal 
logic of Emerson and Clarke [EC82] where time delays are added to the modal operators. TCTL 
can detect only whether a formula is satisfied with probability 0, or with a positive probability, 
or with probability 1. The model checking algorithm transforms a process into a finite state 
process without probabilities and real-time, thus allowing the use of other existing algorithms. 
The problem of model-checking for TCTL is PSPACE-hard. 

2.5.3 Quantitative Method: Model Checking 

Hansson [Han91, Han94] defines a model checking algorithm for his Labeled Concurrent Markov 
Chain model and his branching-time temporal logic TPCTL. Time is discrete in Hansson's 
model, but the logic improves on previous work because probabilities can be quantified (i.e., 
probabilities can be between and 1). The previous model checking algorithms relied heavily 
on the fact that probabilities were not quantified. The algorithm is based on the algorithm 
for model checking of Clarke, Emerson and Sistla [CES83], and on previous work of Hansson 
and Jonsson [HJ89] where a model checking algorithm for PCTL (TPCTL without time) is 
presented. In order to deal with quantified probabilities, the algorithm reduces the computation 
of the probability of an event to a collection of finitely many linear recursive equations. The 
algorithm has an exponential complexity; however, Hansson shows that for a large class of 
interesting problems the algorithm is polynomial. 
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Chapter 3 

Preliminaries 

3.1 Probability Theory 

The rigorous study of randomized algorithms requires the use of several probability measures. 
This section introduces the basic concepts of measure theory that are necessary. Most of the 
results are taken directly from Halmos [Hal50] and Rudin [Rud66], and the proofs can be found 
in the same books or in any other good book on measure theory or probability theory. 

3.1.1 Measurable Spaces 

Consider a set S7. A field on S7, denoted by F, is a family of subsets of S7 that contains 0, and 
that is closed under complementation and finite union. A a- field on 0, denoted by J 7 , is a field 
on S7 that is closed under countable union. The elements of a <7-field are called measurable sets. 
The pair (0, J 7 ) is called a measurable space. 

A field generated by a family of sets C, denoted by F(C), is the smallest field that contains 
C. The (T-field generated by a family of sets C, denoted by cr(C), is the smallest u-field that 
contains C. The family C is called a generator for cr(C). A trivial property of a generator C is 
a(C) = a(F(C)). 

The field generated by a family of sets can be obtained following a simple procedure. 

Proposition 3.1.1 Let C be a family of subsets o/O. 

1. Let Fi(C) be the family containing 0, 0, and all C C J7 such that C G C or (0 — C) G C. 

2. Let i*2(C) ^ e ^ e family containing all finite intersections of elements of F\(C). 

3. Let Fs(C) be the family containing all finite unions of disjoint elements of ^(C). 

Then F(C) = F 3 (C). ■ 

3.1.2 Probability Measures and Probability Spaces 

Let C be a family of subsets of S7. A measure fj, on C is a function that assigns a non-negative 
real value (possibly oo) to each element of C, such that 

1. if is an element of C, then /i(0) = 0. 
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2. if (C'i)i^N forms a sequence of pairwise disjoint elements of C, and U 8 C 8 ' is an element of 
C, then /z(U,-C,-) = £,■/*(<?.■)• 

The last property is called a-additivity. If (0, J 7 ) is a measurable space, then a measure on .F 
as called a measure on (0, J 7 ). 

A measure on a family of sets C is finite if the measure of each element of C is finite. 

A measure space is a triple (0, F, ^), where (0, J 7 ) is a measurable space, and /i is a measure 
on (0, J 7 ). A measure space (0, F, ^) is complete iff for each element C of F such that /u(C) = 0, 
each subset of C is measurable and has measure 0, i.e., for each C" C C , C" G F and /i(C") = 0. 
A measure space is discrete if F is the power set of and the measure of each measurable set 
is the sum of the measures of its points. Discrete spaces will play a fundamental role in our 
theory. 

A probability space is a triple (0, "F, P), where (S7,F) is a measurable space, and P is a 
measure on (S7,F) such that P(0) = 1. The measure P is also referred to as a probability 
measure or a probability distribution. The set is called the sample space, and the elements 
of F are called events. We denote a generic event by P, possibly decorated with primes and 
indices. A standard convention with probability measures and event is that the measure of an 
event is denoted by P[E] rather than by P(E). 

3.1.3 Extensions of a Measure 

The following two theorems shows methods to extend a measure defined on a collection of sets. 
The first theorem says that it is possible to dehne a probability measure P on a measurable 
space (S7,F) by specifying P only on a generator of F; the second theorem states that every 
measure space can be extended to a complete measure space. 

Thus, from the first theorem we derive that in order to check the equality of two probability 
measures Pi and Pi on (0, F), it is enough to compare the two measures on a held that generates 
F. 

Theorem 3.1.2 (Extension theorem) A finite measure fj, on a field F has a unique exten- 
sion to the a-field generated by F . Fhat is, there exists a unique measure jl on o(F) such that 
for each element C of F, J2(C) = /i(C). ■ 

Theorem 3.1.3 Let (S7,F 7/ u) be a measure space. Let J 7 ' be the set of subsets o/O of the form 
C U N such that C G F and N is a subset of a set of measure in F. Fhen, T' is a a-field. 
Furthermore, the function fj,' defined by fJ,'(C U N) = /i(C) is a complete measure on T' . We 
denote the measure space (0, J 7 ', //) by completion((£l, T ', fj,)) . ■ 

3.1.4 Measurable Functions 

Let (S7,.F) and (i7',F') be two measurable spaces. A function y : — > 0' is said to be a 
measurable function from (S7,.F) to (Q 1 ,T r ) if for each set C of T' the inverse image of C, 
denoted by / _1 (C), is an element of T. The next proposition shows that the measurability of 
/ can be checked just by analyzing a generator of T' . 

Proposition 3.1.4 Let (£l,T) and (0' ' ,T') be two measurable spaces, and let C be a generator 
of J 7 '. Let f be a function form to 0'. Fhen f is measurable iff for each element C of C, the 
inverse image / _1 (C) is an element of J 7 . ■ 
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Another property that we need is the closure of measurable functions under composition. 

Proposition 3.1.5 Let f be a measurable function from (Oi, ^i) to (O2, -^"2); ond let g be a 
measurable function from (0,2, J- 2) to (Os,^)- Then fog is a measurable function from (0,\, T\) 
to(0 3 ,T 3 ). m 

3.1.5 Induced Measures and Induced Measure Spaces 

Proposition 3.1.6 Let f be a measurable function from (0, J 7 ) to (0', J 7 '), and let ^ be a 
measure on (0,T). Let // be defined on T' as follows: for each element C of T' ', //(C) = 
//(/ _1 (C)). Then // is a measure on (0', T'). The measure // is called the measure induced by 
f, and is denoted by /(//). ■ 

Based on the result above, it is possible to transform a measure space using a function /. 
Let (0,T,i-i) be a measure space, and let / be a function defined on 0. Let 0' be f(0), and 
let .F' be the set of subsets C of 0' such that / _1 (C) G "F. Then, J 7 ' is a u-field, and / is a 
measurable function from (0, J 7 ) to (0', J 7 '). Thus, the space (0',J-'',f(p)) is a measure space. 
We call such a space the space induced by /, and we denote it by f((0, "F, //)). Observe that 
if (fi,.F, /i) is a probability space, then f((0, "F, //)) is a probability space as well, and that 
induced measure spaces preserve discreteness and completeness. 

3.1.6 Product of Measure Spaces 

Let (0,i, J-i) and (f^,-^) ^ e two measurable spaces. Denote by T\ ® Ti the u-field generated 
by the set of rectangles {C\ X C'2 \ C\ G T\,Ci G T^\. The product space of (Oi, "?i) and 
(i72? ^2)? denoted by (Oi, "?i) ® (i^2? ^2)? is the measurable space (Oi X ^2,-^1 ® ^2)- 

Proposition 3.1.7 Let (0\, J-\, fJ,\) and (i^,-^,/^) ^ e ^ wo measure spaces where //1 and [i^ 
are finite measures. Then there is a unique measure, denoted by [i\ ® [ii, on 3~\ ® T2 such that 
for each C\ G T\ and C 2 G T<i, Mi ® ^(C\ X C 2 ) = /Ui(Ci)//2(C 2 ). ■ 

The product measure space of two measure spaces (fii, "?i,/ii) and (O2? ^2? M2)? denoted by 
(Oi, "Fi,/ii) ® (O2, ^2? ^2)? is the measure space (Oi X ^2,^1 ® Ti,li>\ ® ^2)- It is easy to check 
that if (Oi, T\,ii\) and (Oli^i^'i) are probability spaces, then their product is a probability 
space as well. 

The product of two measure spaces is invertible. Let (0,T ,11) = (fii, "?i,/ii) ® (f^,-^,/^), 
and let 7r 8 -, i = 1,2, be a projection function from Oi X O2 to 0,-, that maps each pair (2:1,2:2) 
to x t . Let fi(- = 7r t -(fi,-), and let T[ = {C \ Trf^C) G -F 8 }. Then (fij-, ^f) = (fi,-,^v), and vr 8 is 
a measurable function from (0,T) to (O'^J 7 -). The measure 7Tj-(/i) coincides with //;, since for 
each C G T\, 7T - " (C) = C X f^, an d for each C <E J~2, ^2 (C) = ^i X C. Thus, the projection 
of (0,T,fi) onto its i th component is (Oi,Ti,[J,i). 

3.1.7 Combination of Discrete Probability Spaces 

In our theory there are several situations in which a discrete probability space is chosen accord- 
ing to some probability distribution, and then an element from the chosen probability space 
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is chosen according to the corresponding probability distribution. The whole process can be 
described by a unique probability space. 

Let {(fi,-, P;, Pi)}i>o be a family of discrete probability spaces, and let {pi}i>o be a family 
of real numbers between and 1 such that J2i>oPi = 1- Define J2i>o(^i^i^i) t° be the triple 
(fi,P, P), where 0, = U 8 >oiP', P = 2 fi , and, for each x G 0, P[x] = J2i>o\ x eQ- Pi-^i[ x ]- ^ i s eas y 
to verify that (0, J 7 , P) is a probability space. 

The process described by (0, P, P) is the following: a probability space (0,-, P 8 -, P 8 ) is drawn 
from {(Oj-,P"j-, P«-)}i>o with probability p 8 -, and then an element x is drawn drom 0,- with prob- 
ability P,-[a;]. 

3.1.8 Conditional Probability 

Let (Q^J-'jP) be a probability space, and let E be an element of P. Frequently, we need to 
study the probability of an event E' of P knowing that event E has occurred. For example, we 
may want to study the probability that a dice rolled 6 knowing that it rolled a number greater 
than 3. The probability of a conditional event is expressed by P[E'\E]. If P[E] = 0, then 
P[E'\E] is undefined; if P[E] > 0, then P[E'\E] is defined to be P[E n E']/P[E]. 

Suppose that P[E] > 0, and consider the triple (ft|P,P|P,P|P) where 0|P = P, P|P = 
{E'HE \ E' e P}, and for each event E' of P|P, P|P[P'] = P[E'\E]. Then it is easy to show 
that (S7|P,P|P,P|P) is a probability space. We call this space a conditional probability space. 

Conditional measures give us an alternative way to express the probability of the intersection 
of several events. That is, 

P^ n • • • n E n ] = PCP^P^lPi] • • -P[P n |Pi n • • • n P n _ x ]. 

If P[E'] = P[E'\E], then P[E n E'] = P[E]P[E']. In this case the events P and E' are said 
to be independent. 

3.1.9 Expected Values 

Let (S7,P) be a measurable space, and let (!R, 1Z) be the measurable space where ?R. is the set 
of real numbers, and 1Z is the u-field generated by the open sets of the real line. A random 
variable on (S7,P), denoted by X, is a measurable function from (S7,P) to (!R, 1Z). 

We use random variables to deal with timed systems. An example of a random variable is 
the function that, given a computation of a system, returns the time it takes to the system to 
achieve a goal in the given computation. In our case, the computations of a system are chosen 
at random, and thus, a natural estimate of the performance of the system is the average time 
it takes to the system to achieve the given goal. 

The above idea is expressed formally by the expected value of a random variable, which is a 
weighted average of X. Specifically, let (0, P, P) be a probability space, and let X be a random 
variable on (S7,P). Then the expected value of X, denoted by P[A], is the weighted average 
of X based on the probability distribution P. We do not show how to compute the expected 
value of a random variable in general, and we refer the interested reader to [Hal50]. Here we 
just mention that if S7 can be partitioned in a countable collection of measurable sets (C 8 ) 8 > 
such that for each set d, X(C'i) is a singleton, then E[X] = J2i>o P[Ci]X(ci), where for each i 
c; is an element of P,- . 
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3.1.10 Notation 

Throughout the thesis we adopt some conventional notation concerning probability spaces. We 
use the notation V, possibly decorated with indexes and primes, to denote a generic probability 
space. Thus, the expression V[ stands for the probability space (0'-, "F/, P/). Furthermore, if 
a generic expression exp denotes a probability space (0, J 7 , P), we use 0, exp ,J- exp , and P exp to 
denote 0, "F, and P, respectively. 

If (fi, "F, P) is a probability space, and E is a generic set, we use P[E] to denote P[E n 0]. 
If E fl is not an element of F, then P[P] is undefined. 

A special kind of probability space is a probability space with a unique element in its sample 
set. The corresponding measure is called a Dime distribution. We use the notation V(x) to 
denote a probability space (0, "F, P) where S7 = {a;}. 

Another important kind of probability space is a space with finitely many elements, each 
one with the same probability. The corresponding measure is called a uniform distribution. 
We use the notation U(x\, . . .,x n ) to denote a discrete probability space (0, "F, P) where S7 = 
{2:1, . . . , x n } and, for each element X{ of 0, P[a; 8 ] = 1/n. 

In the thesis we use heavily discrete probability spaces with no O-probability elements. It 
is easy to verify that the sample set of these probability spaces is at most countable. If C is 
any set, then we denote by Probs(C) the set of discrete probability spaces (0, "F, P) with no 
O-probability elements such that S7 C C . 

3.2 Labeled Transition Systems 

A Labeled Transition System [Kel76, Plc>81] is a state machine with labeled transitions. The 
labels, also called actions, are used to model communication between a system and its external 
environment. Labeled transition systems have been used successfully for the analysis of con- 
current and distributed systems [DH84, Mil89, LT87, LV93a]; for this reason we choose them 
as our basic model. 

Currently there are several definitions of labeled transition systems, each one best suited 
for the kind of application it is meant for. In this section we present a definition of labeled 
transition systems in the style of [LV93a]. 

3.2.1 Automata 

An automaton A consists of four components: 

1. a set states(A) of states. 

2. a nonempty set start(A) C states(A) of start states. 

3. an action signature sig(A) = (ext(A), int(A)), where ext(A) and int(A) are disjoint sets 
of external and internal actions, respectively. Denote by acts(A) the set ext(A) U int(A) 
of actions. 

4. a transition relation trans(A) C states(A) X acts(A) X states(A). The elements of trans(A) 
are referred to as transitions or steps. 
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Figure 3-1: The Buffer automaton. 

Thus, an automaton is a labeled transition system, possibly with multiple start states, whose 
actions are partitioned into external and internal actions. The external actions model com- 
munication with the external environment; the internal actions model internal communication, 
not visible from the external environment. 

We use s to denote a generic state, and a and b to denote a generic action. We also use r to 
denote a generic internal action. All our conventional symbols may be decorated with primes 
and indexes. We say that an action a is enabled from a state s in A if there exists a state s' of 
A such that (s, a, s') is a transition of A. 

A standard alternative notation for transitions is s — ► s' . This notation can be extended to 
finite sequences of actions as follows: s - — " s' iff there exists a sequence of states si, . . . , s n _i 
such that s —^ si — ^-> • • -s n _i — -^ s n . To abstract from internal computation, there is another 
standard notion of weak transition, denoted by s =>■ s' . The action a must be external, and 
the meaning of s =>■ s' is that there are two finite sequences /3i,/?2 of internal actions such that 

s — > s' . As for ordinary transitions, weak transitions can be generalized to finite sequences 
of external actions. A special case is given by the empty sequence: s =>■ s' iff either s' = s or 

there exists a finite sequence (3 of internal actions such that s — > s' . 

Example 3.2.1 A classic example of an automaton is an unbounded ordered buffer that stores 
natural numbers (see Figure 3-1). An external user sends natural numbers to the buffer, and 
the buffer sends back to the external environment the ordered sequence of numbers it receives 
from the user. 

The automaton Buffer of Figure 3-1 can be described as follows. All the actions of Buffer 
are external and are of the form insert(i) and extract(i), where i is a natural number, i.e., the 
actions of Buffer are given by the infinite set U 8 gjv{msert(i), extract(i)} . The states of Buffer 
are the finite sequences of natural numbers, and the start state of Buffer is the empty sequence. 
The actions of the form insert (i) are enabled from every state of Buffer, i.e., for each state 
s and each natural number i there is a transition (s, insert(i),is) in Buffer, where is denotes 
the sequence obtained by appending i to the left of s. The actions of the form extract(i) are 
enabled only from those states where i is the rightmost element in the corresponding sequence 
of numbers, i.e., for each state s and each natural number i there is a transition (si, extract(i), s) 
of Buffer. No other transitions are defined for Buffer. 

Observe that from every state of Buffer there are infinitely many actions enabled. The 
way to choose among those actions is not specified in Buffer. In other words, the choice of the 
transition to perform is nondeterministic. In this case the nondeterminism models the arbitrary 
behavior of the environment. 
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Figure 3-2: Concatenation of two buffers. 

The role of internal actions becomes clear when we concatenate two buffers as in Figure 3-2. 
The communication that occurs between the two buffers is internal in the sense that it does not 
affect directly the external environment. Another useful observation about the concatenation 
of the two buffers in Figure 3-2 is that nondeterminism expresses two different phenomena: the 
arbitrary behavior of the environment, and the arbitrary scheduling policy that can be adopted 
in choosing whether Buffer 1 or Buffer 2 performs the next transition. In general nondeterminism 
can express even a third phenomenon, namely, the fact that an arbitrary state can be reached 
after the occurrence of an action. Such a form of nondeterminism would arise if we assume that 
a buffer may lose data by failing to modify its state during an insertion operation. ■ 

3.2.2 Executions 

The evolution of an automaton can be described by means of its executions. An execution 
fragment a of an automaton A is a (finite or infinite) sequence of alternating states and actions 
starting with a state and, if the execution fragment is finite, ending in a state 

a = So«lSl«2 s 2 ■ ■ ■ 

where for each i, (s 8 -, a 8 +i, s;+i) is a transition of A. Thus, an execution fragment represents a 
possible way to resolve the nondeterminism in an automaton. 

Denote by fstate(a) the first state of a and, if a is finite, denote by Istate(a) the last state of 
a. Furthermore, denote by frag* (A) and frag(A) the sets of finite and all execution fragments 
of A, respectively. 

An execution is an execution fragment whose first state is a start state. Denote by exec* (A) 
and exec(A) the sets of finite and all execution of A, respectively. A state s of A is reachable if 
there exists a finite execution of A that ends in s. 

The length of an execution fragment a, denoted by \a\, is the number of actions that occur 
in a. If a is infinite, then \a\ = oo. 

A finite execution fragment a\ = so a i s i ■■■a n s n of A and an execution fragment a.^ = 
s n a n+ is n+ i ■ ■ ■ of A can be concatenated . In this case the concatenation, written a\ ~ a.^, is 
the execution fragment so a i s i • • ■ a n s n a n+ is n+ i ■ ■ ■. If a = a\ ~ a.^, then we denote a.^ by a>a\ 
(read "a after «i"). 

An execution fragment a\ of A is a prefix of an execution fragment a.^ of A, written a\ < ci2, 
if either a\ = a.^ or a.\ is finite and there exists an execution fragment a[ of A such that 
ci2 = a.\ ~ a[. The execution fragment a[ is also called a suffix of a 2 and is denoted by a.2 >a i- 
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3.2.3 Traces 

The executions of an automaton contain a lot of information that is irrelevant to the environ- 
ment, since the interaction between an automaton and its environment occurs through external 
actions only. The trace of an execution is the object that represents the actual interaction that 
occurs between an automaton and its environment during an execution. 

The trace of an execution (fragment) a of an automaton A, written trace a(oi), or just 
trace(a) when A is clear, is the list obtained by restricting a to the set of external actions of 
A, i.e., trace(a) = a \ ext(A). We say that (3 is a trace of an automaton A if there exists an 
execution a of A with trace(a) = (3. Denote by traces*(A) and traces(A) the sets of finite and 
all traces of A, respectively. Note, that a finite trace can be the trace of an infinite execution. 

3.2.4 Trace Semantics 

In [LV93a] automata are compared based on traces. Specifically, a preorder relation is defined 
between automata based on inclusion of their traces: 

A\ Cy Ai iff traces(A\) C traces(A2). 

The trace preorder can express a notion of implementation, usually referred to as a safe imple- 
mentation. That is, A\, the implementation, cannot do anything that is forbidden by A2, the 
specification. For example, no implementation of the buffer of Figure 3-1 can return natural 
numbers that were never entered or natural numbers in the wrong order. 

Although the trace preorder is weak as a notion of implementation, and so finer relations 
could be more appropriate [DeN87, Gla90, Gla93], there are several situations where a trace 
based semantics is sufficient [LT87, Dil88, AL93, GSSL94]. The advantage of a trace based 
semantics is that it is easy to handle. 

In this thesis we concentrate mainly on trace based semantics; however, the techniques that 
we develop can be extended to other semantic notions as well. 

3.2.5 Parallel Composition 

Parallel composition is the operator on automata that identifies how automata communicate 
and synchronize. There are two main synchronization mechanisms for labeled transition sys- 
tems, better known as the CCS synchronization style [Mil89], and the CSP synchronization 
style [Hoa85]. In the CCS synchronization style the external actions are grouped in pairs of 
complementary actions; a synchronization occurs between two automata that perform comple- 
mentary actions, and becomes invisible to the external environment, i.e., a synchronization is 
an internal action. Unless specifically stated through an additional restriction operator, an 
automaton is allowed not to synchronize with another automaton even though a synchroniza- 
tion is possible. In the CSP synchronization style two automata must synchronize on their 
common actions and evolve independently on the others. Both in the CCS and CSP styles, 
communication is achieved through synchronization. 

In this thesis we adopt the CSP synchronization style, which is essentially the style adopted 
in [LT87, Dil88, LV93a]. A technical problem that arises in our framework is that automata 
may communicate through their internal actions, while internal actions are not supposed to be 
visible. To avoid these unwanted communications, we define a notion of compatibility between 
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automata. Two automata A\,A 2 are compatible iff int(A\) n acts(A 2 ) = and acts(A\) n 
int(A 2 ) = 0. 

The parallel composition of two compatible automata A\,A 2 , denoted by Ai||A 2 , is the 
automaton A such that 

1. states(A) = states(A\) X states(A 2 ). 

2. start(A) = sfarf(Ai) X start(A 2 ). 

3. si'ff(A) = (eatf(Ai) U ext(A 2 ), mf(Ai) U mf(A 2 )). 

4. ((si,s 2 ),a, (si,^)) e ^ans(A) iff 

(a) if a G acfs(Ai), then (si,a, s^) G irans(Ai), else s[ = si, and 

(b) if a G ac£s(A 2 ), then (s2,a,s 2 ) £ trans(A 2 ), else S2 = s 2 . 

If two automata are incompatible and we want to compose them in parallel, the problem 
can be solved easily by renaming the internal actions of one of the automata. The renaming 
operation is simple: just rename each occurrence of each action in the action signature and the 
transition relation of the given argument automaton. At this point it is possible to understand 
how to build a system like the one described in Figure 3-2. Buffer 1 is obtained from Buffer by 
renaming the actions extract(i) into t(i), and Buffer 2 is obtained from Buffer by renaming the 
actions insert (i) into t(i). Then, Buffer \ and Buffer 2 are composed in parallel, and finally the 
actions t(i) are made internal. This last step is achieved through a Hide operation, whose only 
effect is to change the signature of an automaton. 

We conclude by presenting two important properties of parallel composition. The first 
property concerns projections of executions. Let A = A\\\A 2 , and let (si,s 2 ) be a state of A. 
Let i be either 1 or 2. The projection of (si,s 2 ) onto Ai, denoted by (si,s 2 )\Ai, is s 8 -. Let 
a = so«iSi • • • be an execution of A. The projection of a onto Ai, denoted by a\Ai is the 
sequence obtained from a by projecting all the states onto Ai, and by removing all the actions 
not in acts(Ai) together with their subsequent states. 

Proposition 3.2.1 Let A = A\\\A 2 , and let a be an execution of A. Then a\A\ is an execution 
of A\ and a \A 2 is an execution of A 2 . ■ 

The projection of an execution of A onto one of the components Ai is essentially the view of 
Ai of the execution a. In other words the projection represents what Ai does in order for A to 
produce a. Proposition 3.2.1 states that the view of Ai is indeed something that Ai can do. 
The second property concerns the trace preorder. 

Proposition 3.2.2 Let A\ Cy A[. Then, for each A 2 compatible with both A\ and A[, 

Ai\\A 2 QtA[\\A 2 . ■ 

The property expressed in Proposition 3.2.2 is better known as substitutivity or compositionality . 
In other words Cy is a precongruence with respect to parallel composition. Substitutivity is one 
of the most important properties that an implementation relation should satisfy. Informally, 
substitutivity says that an implementation A\ of a system A[ works correctly in any context 
where A[ works correctly. Substitutivity is also the key idea at the base of modular verification 
techniques. 
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Chapter 4 

Probabilistic Automata 



4.1 What we Need to Model 

Our main goal is to analyze objects that at any point can evolve according to a probability 
distribution. The simplest example of a random computation is the process of flipping a coin. 
Thus, a program may contain an instruction like 

x := flip 

whose meaning is to assign to x the result of a coin flip. From the state-machine point of view, 
the transition relation of the corresponding automaton should be specified by giving the states 
reachable after the coin flip, together with their probability. Thus, the coin flipping process 
can be represented by the labeled transition system of Figure 4-1. The edges joining two states 
are associated with an action and a weight, where the weight of an edge is the probability of 
choosing that specific edge. Thus, we require that for each state that has some outgoing edges, 
the sum of the weights of the outgoing edges is 1. 

However, we also need to deal with nondeterminism. Consider a more complicated process 
where a coin is flipped, but where the coin can be either fair, i.e., it yields head with probability 
1/2, or unfair by yielding head with probability 2/3. Furthermore, suppose that the process 
emits a beep if the result of the coin flip is head. In this case, the choice of which coin to flip 
is nondeterministic, while the outcome of the coin flip is probabilistic. The start state should 
enable two separate transitions, each one corresponding to the flip of a specific coin. Figure 4- 
2 represents the nondeterministic coin flipping process. The start state enables two separate 
groups of weighted edges; each group is identified by an arc joining all of its edges, and the 
edges of each group form a probability distribution. 

At this point we may be tempted to ask the following question: 

head 




' tail 
Figure 4-1: The coin flipping process. 
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Figure 4-2: The nondeterministic coin flipping process. 

"What is the probability that the nondeterministic coin flipper beepsV 

The correct answer is 

"It depends on which coin is flipped." 

Although this observation may appear to be silly, the lesson that we learn is that it is not 
possible to talk about the probability of some event until the nondeterminism is resolved. 
Perhaps we could give a more accurate answer as follows: 

"The probability that the nondeterministic coin flipper beeps is either 1/2 or 2/3, 
depending on which coin is flipped." 

However, there are two possible objections. The first objection concerns the way a coin is 
chosen. What happens if the coin to be flipped is chosen at random? After all, in the definition 
of the nondeterministic coin flipper there are no limitations to the way a coin is chosen. In this 
case, the correct answer would be 

"The probability that the nondeterministic coin flipper beeps is between 1/2 and 2/3, 
depending on how the coin to be flipped is chosen." 

The second objection concerns the possibility of scheduling a transition. What happens if the 
scheduler does not schedule the beep transition even though it is enabled? In this case the 
correct answer would be 

" Under the hypothesis that some transition is scheduled whenever some transition is 
enabled, the probability that the nondeterministic coin flipper beeps is between 1/2 
and 2/3, depending on how the coin to be flipped is chosen." 

There is also another statement that can be formulated in relation to the question: 

" The nondeterministic coin flipper does not beep with any probability greater than 

2/3." 

This last property is better known as a safety property [AS85] for ordinary labeled transition 
systems. 

Let us go back to the scheduling problem. There are actual cases where it is natural to allow 
a scheduler not to schedule any transition even though some transition is enabled. Consider a 
new nondeterministic coin flipper with two buttons, marked fair and unfair, respectively. The 
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Figure 4-3: The triggered coin flipping process. 




Figure 4-4: A computation of the triggered coin flipping process. 

buttons can be pressed by an external user. Suppose that pressing one button disables the 
other button, and suppose that the fair coin is flipped if the button marked fair is pressed, 
and that the unfair coin is flipped if the button marked unfair is pressed. The new process 
is represented in Figure 4-3. In this case the scheduler models the external environment, and 
a user may decide not to press any button, thus not scheduling any transition from so even 
though some transition is enabled. An external user may even decide to flip a coin and press 
a button only if the coin gives head, or flip a coin and press fair if the coin gives head and 
press unfair if the coin gives tail. That is, an external user acts like a scheduler that can use 
randomization for its choices. If we ask again the question about the probability of beeping, a 
correct answer would be 

"Assuming that beep is scheduled whenever it is enabled, the probability that the 
triggered coin flipper beeps, conditional to the occurrence of a coin flip, is between 
1/2 and 2/3." 

Suppose now that we resolve all the nondeterminism in the triggered coin flipper of Figure 4-3, 
and consider the case where the external user presses fair with probability 1/2 and unfair 
with probability 1/2. In this case it is possible to study the exact probability that the process 
beeps, which is 7/12. Figure 4-4 gives a representation of the outcome of the user we have just 
described. Note that the result of resolving the nondeterminism is not a linear structure as is 
the case for standard automata, but rather a tree-like structure. This structure is our notion 
of a probabilistic execution and is studied in more detail in Section 4.2. 
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4.2 The Basic Model 

In this section we introduce the basic probabilistic model that is used in the thesis. We formalize 
the informal ideas presented in Section 4.1, and we extend the parallel composition operator 
of ordinary automata to the new framework. We also introduce several notational conventions 
that are used throughout the thesis. 

4.2.1 Probabilistic Automata 

A probabilistic automaton M consists of four components: 

1. A set states(M) of states. 

2. A nonempty set start(M) C states(M) of start states. 

3. An action signature sig(M) = (ext(M), int(M)), where ext(M) and int(M) are disjoint 
sets of external and internal actions, respectively. Denote by acts(M) the set ext(M) U 
int(M) of actions. 

4. A transition relation trans(M) C states(M) X Probs((acts(M) X states(M))L){8}). Recall 
from Section 3.1.10 that for each set C, Probs(C) denotes the set of discrete probability 
spaces (S7,.F, P) with no 0-probability elements such that S7 C C. The elements of 
trans(M) are referred to as transitions or steps. 

A probabilistic automaton differs from an ordinary automaton only in the transition relation. 
Each transition represents what in the figures of Section 4.1 is represented by a group of edges 
joined by an arc. From each state s, once a transition is chosen nondeterministically, the 
action that is performed and the state that is reached are determined by a discrete probability 
distribution. Each transition (s,V) may contain a special symbol 8, which represents the 
possibility for the system not to complete the transition, i.e., to remain in s without being able 
to engage in any other transition. 

Example 4.2.1 (Meaning of 6) To give an idea of the meaning of 8, suppose that M models 
a person sitting on a chair that stands up with probability 1/2. That is, from the start state so 
there is a transition of M where one outcome describes the fact that the person stands up and 
the other outcome describes the fact that the person does not stand up (this is 8). The point 
is that there is no instant in time where the person decides not to stand up: there are only 
instants where the person stands up. What the transition leaving so represents is that overall 
the probability that the person does the action of standing up is 1/2. The need for 8 is clarified 
further in Section 4.2.3, where we study probabilistic executions, and in Section 4.3, where we 
study parallel composition. ■ 

The requirement that the probability space associated with a transition be discrete is imposed 
to simplify the measure theoretical analysis of probabilistic automata. In this thesis we work 
with discrete probability spaces only, and we defer to further work the extension of the theory 
to more general probability spaces. The requirement that each transition does not lead to any 
place with probability is imposed to simplify the analysis of probabilistic automata. All the 
results of this thesis would be valid even without such a restriction, although the proofs would 
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contain a lot of uninteresting details. The requirement becomes necessary for the study of live 
probabilistic automata, which we do not study here. 

There are two classes of probabilistic automata that are especially important for our analysis: 
simple probabilistic automata, and fully probabilistic automata. 

A probabilistic automaton M is simple if for each transition (s,V) of trans(M) there is an 
action a of M such that S7 C {a} X states(M). In such a case, a transition can be represented 
alternatively as (s,a,V'), where V' G Probs(states(M)), and it is called a simple transition with 
action a. The probabilistic automata of Figures 4-2 and 4-3 are simple. In a simple probabilistic 
automaton each transition is associated with a single action and it always completes. The idea 
is that once a transition is chosen, then only the next state is chosen probabilistically. In 
this thesis we deal mainly with simple probabilistic automata for a reason that is made clear 
in Section 4.3. We use general probabilistic automata to analyze the computations of simple 
probabilistic automata. 

A probabilistic automaton M is fully probabilistic if M has a unique start state, and from 
each state of M there is at most one transition enabled. Thus, a fully probabilistic automaton 
does not contain any nondeterminism. Fully probabilistic automata play a crucial role in the 
definition of probabilistic executions. 

Example 4.2.2 (Probabilistic automata) A probabilistic Turing Machine is a Turing ma- 
chine with an additional random tape. The content of the random tape is instantiated by 
assigning each cell the result of an independent fair coin flip (say if the coin gives head and 
1 if the coin gives tail). If we assume that each cell of the random tape is instantiated only 
when it is reached by the head of the machine, then a probabilistic Turing machine can be 
represented as a simple probabilistic automaton. The probabilistic automaton, denoted by M, 
has a unique internal action r, and its states are the instantaneous descriptions of the given 
probabilistic Turing machine; each time the Turing machine moves the head of its random tape 
on a cell for the first time, M has a probabilistic transition that represents the result of reaching 
a cell whose content is with probability 1/2 and 1 with probability 1/2. 

An algorithm that at some point can flip a coin or roll a dice can be represented as a simple 
probabilistic automaton where the flipping and rolling operations are simple transitions. If the 
outcome of a coin flip or dice roll affects the external behavior of the automaton, then the 
flip and roll actions can be followed by simple transitions whose actions represent the outcome 
of the random choice. Another possibility is to represent the outcome of the random choice 
directly in the transition where the random choice is made by performing different actions. In 
this case the resulting probabilistic automaton would not be simple. Later in the chapter we 
show why we prefer to represent systems as simple probabilistic automata when possible. ■ 

4.2.2 Combined Transitions 

In Section 4.1 we argued that a scheduler may resolve the nondeterminism using randomization, 
i.e., a scheduler can generate a new transition by combining several transitions of a probabilistic 
automaton M. We call the result of the combination of several transitions a combined transition. 
Formally, let M be a probabilistic automaton, and let s be a state of M. Consider a finite or 
countable set {(s,Vi)}i£i of transitions of M leaving from s, and a family of non- negative 
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weights {pi}i e i such that J2iPi ^ 1- Let 



e/|k>o / V iei I 



V = >; P,-7>,-| + |l->>,-|2>(«), (4.1) 

i.e., P is a combination of discrete probability spaces as described in Section 3.1.7. The 
pair (s,V) is called a combined transition of M and is denoted by ^2i^iPi(s,Vi). Denote 
by ctrans(M) the set of combined transitions of M. Note that trans(M) C ctrans(M). 

Thus, the combination of transitions can be viewed as a weighted sum of transitions where 
the sum of the weights is at most 1. If the sum of the weights is not 1, then nothing is 
scheduled by default. The reason for 8 by default will become clear when we analyze parallel 
composition in Section 4.3. Note that all the transitions (s,Vi) where pt = are discarded in 
Expression (4.1), since otherwise V would contain elements whose probability is 0. We do not 
impose the restriction that each pi is not for notational convenience: in several parts of the 
thesis the p 8 's are given by complex expression that sometimes may evaluate to 0. 

Proposition 4.2.1 The combination of combined transitions of a probabilistic automaton M 
is a combined transition of M . 

Proof. Follows trivially from the definition of a combined transition. ■ 

4.2.3 Probabilistic Executions 

If we resolve both the nondeterministic and probabilistic choices of a probabilistic automaton, 
then we obtain an ordinary execution like those usually defined for ordinary automata. Thus, an 
execution fragment of a probabilistic automaton M is a (finite or infinite) sequence of alternating 
states and actions starting with a state and, if the execution fragment is finite, ending in a state, 

a = So«lSl«2 s 2 • • • 5 

where for each i there is a transition (sj-,7-\-+i) of M such that (a;+i, Si+i) G ^;+i- Executions, 
concatenations of executions, and prefixes can be defined as for ordinary automata. 

In order to study the probabilistic behavior of a probabilistic automaton, we need a mech- 
anism to resolve only the nondeterminism, and leave the rest unchanged. That is, we need a 
structure that describes the result of choosing a transition, possibly using randomization, at 
any point in history, i.e., at any point during a computation. In Figure 4-4 we have given an 
example of such a structure, and we have claimed that it should look like a tree. Here we give 
a more significant example to justify such a claim. 

Example 4.2.3 (History in a probabilistic execution) Consider a new triggered coin flip- 
per, described in Figure 4-5, that can decide nondeterministically to beep or boo if the coin flip 
yields head, and consider a computation, described in Figure 4-6, that beeps if the user chooses 
to flip the fair coin, and boos if the user chooses to flip the unfair coin. Then, it is evident that 
we cannot identify the two states head of Figure 4-6 without reintroducing nondeterminism. In 
other words, the transition that is scheduled at each point depends on the past history of the 
system, which is represented by the position of a state in the tree. For a formal definition of a 
structure like the one of Figure 4-6, however, we need to refer explicitly to the past history of 
a system. ■ 
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Figure 4-5: The triggered coin flipper with a boo sound. 




Figure 4-6: A computation of the triggered coin flipper with a boo sound. 

Let a be a finite execution fragment of a probabilistic automaton M. Define a function a" 
that applied to a pair (a,s) returns the pair (a,aas), and applied to 8 returns 8. Recall from 
Section 3.1.5 that the function a" can be extended to probability spaces. Informally, if (s,V) is 
a combined transition of M and a is a finite execution fragment of M such that Istate(a) = s, 
then the pair (a, a^V) denotes a transition of a structure that in its states remembers part of 
the past history. A probabilistic execution fragment of a probabilistic automaton M, is a fully 
probabilistic automaton, denoted by H , such that 

1. states(H) C frag*(M). Let q range over states of probabilistic execution fragments. 

2. for each transition (q,V) of H there is a combined transition (lstate(q),V r ) of M, called 
the corresponding combined transition, such that V = q ~ 7-*'. 

3. each state q of if is reachable in H and enables one transition, possibly (q,V(8)). 

A probabilistic execution is a probabilistic execution fragment whose start state is a start state of 
M. Denote by prfrag(M) the set of probabilistic execution fragments of M, and by prexec(M) 
the set of probabilistic executions of M. Also, denote by q$ the start state of a generic 
probabilistic execution fragment H . 

Thus, by definition, a probabilistic execution fragment is a probabilistic automaton itself. 
Condition 3 is technical: reachability is imposed to avoid useless states in a probabilistic exe- 
cution fragment; the fact that each state enables one transition is imposed to treat uniformly 
all the points where it is possible not to schedule anything. Figures 4-6 and 4-7 represent 
two probabilistic executions of the triggered coin flipper of Figure 4-5. The occurrence of 8 
is represented by a dashed line labeled with 8. The states of the probabilistic executions are 



49 



Figure 4-7: A probabilistic execution of the triggered coin flipper. 

not represented as finite execution fragments since their position in the diagrams gives enough 
information. Similarly, we omit writing explicitly all the transitions that lead to V(8) (e.g., 
states si and S2 in Figure 4-7). 

We now have enough structure to understand better the role of 8. In ordinary automata a 
scheduler has the possibility not to schedule anything at any point, leading to a finite execution. 
Such assumption is meaningful if the actions enabled from a given state model some input 
that comes from the external environment. In the probabilistic framework it is also possible to 
schedule no transition from some point. Since a scheduler may use randomization in its choices, 
it is also possible that from some specific state nothing is scheduled only with some probability 
p, say 1/2. 

Example 4.2.4 (The role of 6) In the triggered coin flipper of Figure 4-5 a user can flip 
a fair coin to decide whether to push a button, and then, if the coin flip yields head, flip 
another coin to decide which button to press. In the transition that leaves from so we need 
some structure that represents the fact that nothing is scheduled from so with probability 1/2: 
we use 8 for this purpose. Figure 4-7 represents the probabilistic execution that we have just 
described. ■ 

Since a probabilistic execution fragment is itself a probabilistic automaton, it is possible to 
talk about the executions of a probabilistic execution fragment, that is, the ways in which the 
probabilistic choices can be resolved in a probabilistic execution fragment. However, since at 
any point q it is possible not to schedule anything, if we want to be able to study the probabilistic 
behavior of a probabilistic execution fragment then we need to distinguish between being in q 
with the possibility to proceed and being in q without any possibility to proceed. For example, 
in the probabilistic execution of Figure 4-7 we need to distinguish between being in so before 
performing the transition enabled from so and being in so after performing the transition. We 
represent this second condition by writing so<*>. In general, we introduce a notion of an extended 
execution fragment, which is used in Section 4.2.5 to study the probability space associated with 
a probabilistic execution. 

An extended execution (fragment) of a probabilistic automaton M, denoted by a, is either 
an execution (fragment) of M, or a sequence a' 8, where a' is a finite execution (fragment) of 
M, The sequences so<*> and sofairsi8 are examples of extended executions of the probabilistic 
execution of Figure 4-7. 

There is a close relationship between the extended executions of a probabilistic automaton 
and the extended executions of one of its probabilistic execution fragments. Here we define 
two operators that make such a relationship explicit. Let M be a probabilistic automaton and 
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let if be a probabilistic execution fragment of M. Let go be the start state of H . For each 
extended execution a = go«igi • • • of if , let 

a J go ~ lstate(qo)ailstate(qi)a2 • • • if a does not end in 6, , . 

[ go ~ lstate(qo)ailstate(qi)a2- ■ ■ a n lstate(q n )b if a = qoaiqi ■■■ a n q n 8 . 

It is immediate to observe that a{ is an extended execution fragment of M. For each extended 
execution fragment a of M such that go < «, i.e., a = go ~ so a i s i • • •, let 

. a I go«i(gn ~ •So a i s i) a 2(o , o ~ so a i s i a 2 s 2) • • • if a does not end in 8, /,„s 

[ qoaiiqo so«iSi) • • -(go so«iSi •••««•§«)£' if a = g so«iSi • • -a n s n d. 

It is immediate to observe that a|go is an extended execution of some probabilistic execution 
fragment of M. Moreover, the following proposition holds. 

Proposition 4.2.2 Let H be a probabilistic execution fragment of a probabilistic automaton 
M, and let go be the start state of H . Then, for each extended execution a of H , 

(a.{)Uo = a, (4.4) 
and for each extended execution fragment a of M starting with go, 

(aUo)i = a. (4.5) 

Proof. Simple analysis of the definitions. ■ 

The bottom line is that it is possible to talk about extended executions of H by analyzing only 
extended execution fragments of M. 

4.2.4 Notational Conventions 

For the analysis of probabilistic automata and of probabilistic executions we need to refer to 
explicit objects like transitions or probability spaces associated with transitions. In this section 
we give a collection of notational conventions that ease the identification of each object. 

Transitions 

We denote a generic transition of a probabilistic automaton by tr, possibly decorated with 
primes and indices. For each transition tr = (s,V), we denote V alternatively by Vt r . If tr is a 
simple transition, represented by (s, a, i 7 ), we abuse notation by denoting V by Vtr as well. The 
context will always clarify the probability space that we denote. If (s,V) is a transition, we use 
any set of actions V to denote the event {(a, s') G | a £ V} that expresses the occurrence of 
an action from V in V, and we use any set of states U to denote the event {(a, s') G fi | s' G U} 
that expresses the occurrence of a state from U in V . We drop the set notation for singletons. 
Thus, P[a] is the probability that action a occurs in the transition (s,V). 

If M is a fully probabilistic automaton and s is a state of M, then we denote the unique 
transition enabled from s in M by trf 1 ", and we denote the probability space that appears in 
tr^ by Vg . Thus, (rf = (s,Vg). We drop M from the notation whenever it is clear from 
the context. This notation is important to handle probabilistic execution fragments. 
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Transition Prefixing and Suffixing 

Throughout the thesis we use transitions of probabilistic automata and transitions of proba- 
bilistic execution fragments interchangeably. If if is a probabilistic execution fragment of a 
probabilistic automaton M, then there is a strong relation between the transitions of H and 
some of the combined transitions of M. We exploit such a correspondence through two oper- 
ations on transitions. The first operation is called transition prefixing and adds some partial 
history to the states of a transition; the second operation is called transition suffixing and re- 
moves some partial history from the states of a transition. These operations are used mainly 
in the proofs of the results of this thesis. 

Let tr = (s,V) be a combined transition of a probabilistic automaton M, and let a be a 
finite execution fragment of M such that Istate(a) = s. Then the transition a~ tr is defined to 
be (a, a" V). We call the operation oT transition prefixing. 

Let tr = (q,V) be a transition of a probabilistic execution fragment H , and let q' < q. Let 
>q' be a function that applied to a pair (a,q") of returns (a,q">q'), and applied to 8 returns 
8. Let V>q' denote the result of applying >q' to V . Then the transition tr\>q' is defined to be 
(q>q',V>q'). We call the operation >q' transition suffixing. 

The following properties concern distributivity of transition prefixing and suffixing with 
respect to combination of transitions. 

Proposition 4.2.3 Let M be a probabilistic automaton, and let q be a finite execution fragment 
ofM. 

1. q" J2iPi^ r i = J2iPi( a ^ t r 'i)} where each tr\ is a transition of M . 

2. J2iPi^ r i >a = J2iPi(t r i >c l)> where each tr\ is a transition of some probabilistic execution 
fragment of M . 

Proof. Simple manipulation of the definitions. ■ 

4.2.5 Events 

At this point we need to define formally how to compute the probability of some event in 
a probabilistic execution. Although it is intuitively simple to understand the probability of 
a finite execution to occur, it is not as intuitive to understand how to deal with arbitrary 
properties. A probabilistic execution can be countably branching, and can have uncountably 
many executions. As an example, consider a probabilistic execution that at any point draws a 
natural number n > with probability 1/2™. What is measurable? What is the probability of 
a generic event? 

In this section we define a suitable probability space for a generic probabilistic execution 
fragment if of a probabilistic automaton M. Specifically, given a probabilistic execution frag- 
ment H we define a probability space Vh as the completion of another probability space V' H 
which is defined as follows. Define an extended execution a of if to be complete iff either a 
is infinite or a = a' 8 and 8 £ ^utate(a')' Then, the sample space fl' H is the set of extended 
executions of M that originate from complete extended executions of if, i.e., 

£l' H = {a J, | a is a complete extended execution of H}. (4-6) 
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The occurrence of a finite extended execution a of M can be expressed by the set 

C* = {a'ett' H \a< a'}, (4.7) 

called a cone. We drop H from C^ whenever it is clear from the context. Let Ch be the set of 
cones of H . Then define T H to be the u-field generated by Cff , i-e., 

T'u = °(Ch). (4.8) 

To define a probability measure on T' H , we start by defining a measure /iff on Ch such that 
/iij (Oij) = 1. Then we show that /iff can be extended uniquely to a measure /iff on F(Ch), 
where F(Ch) is built according to Proposition 3.1.1. Finally we use the extension theorem 
(Theorem 3.1.2) to show that /iff can be extended uniquely to a probability measure P H on 
ct(F(C h )) = cr(Cff). 

The measure /iff (C„ ) of a cone C„ is the product of the probabilities associated with each 
edge that generates a in H . Formally, let go be the start state of H . If a < qo, then 

/iff(Cf) ^ 1; (4.9) 

if a = q ~ s aisi • • -s n _ia n s n , then 

/iff(Cf) = <[(ai,gi)]---P g UK,g.)], (4.10) 

where for each i, 1 < i < n, qi = qo ~ so a i s i • • -Si-iftiSi; if a = qo ^ so a i s i • • • s n-i a n s n^ then 

Mff(C^) = P%[("i,qi)]---PLM a n><ln)] p M ( 4 - n ) 

where for each i, 1 < i < n, qi = qo ~ so a i s i • • -Si-iftiSi. 

Example 4.2.5 (Some commonly used events) Before proving that the construction of 
V'jj is correct, we give some examples of events. The set describing the occurrence of an action 
a (eventually a occurs) can be expressed as a union of cones of the form C a such that a appears 
in a. Moreover, any union of cones can be described as a union of disjoint cones (follows from 
Lemma 4.2.4 below). Since a probabilistic execution fragment is at most countably branching, 
the number of distinct cones in Ch is at most countable, and thus the occurrence of a can be 
expressed as a countable union of disjoint cones, i.e., it is an event of T H . More generally, any 
arbitrary union of cones is an event. We call such events finitely satisfiable. The reason for the 
word "satisfiable" is that it is possible to determine whether an execution a of 0^ is within a 
finitely satisfiable event by observing just a finite prefix of a. That finite prefix is sufficient to 
determine that the property represented by the given event is satisfied. 

The set describing the non-occurrence of an action a is also an event, since it is the comple- 
ment of a finitely satisfiable event. Similarly, the occurrence, or non-occurrence, of any finite 
sequence of actions is an event. For each natural number n, the occurrence of exactly n a's is 
an event: it is the intersection of the event expressing the occurrence of at least n a's and the 
event expressing the non-occurrence of n + 1 a's. Finally, the occurrence of infinitely many a's 
is an event: it is the countable intersection of the events expressing the occurrence of at least i 
a's, i > 0. ■ 



53 



We now move to the proof that V' H is well defined. First we use ordinal induction to show that 
the function /i# defined on Ch is cr-additive, and thus that /i# is a measure on Ch (Lemma 4.2.6); 
then we show that there is a unique extension of /i# to F(Ch) (Lemmas 4.2.7, 4.2.8, and 4.2.9). 
Finally, we use the extension theorem to conclude that P' H is well defined. 

Lemma 4.2.4 Let C ai ,C a2 £ !)#. If ot\ < a.^ then C ai C C a2 . If ai ^ a.^ and a.^ ^ a.\ then 

c ai n c a2 = 0. 

Proof. Simple analysis of the definitions. ■ 

Lemma 4.2.5 Let H be a probabilistic execution of a probabilistic automaton M , and let q be 
a state of LI . Suppose that there is a transition enabled from q in H . Lhen 

m(Ljq) ~ \ E(a,,0€Of M<v) + Mc,«) ./ se nf . l4 ^ j 

Proof. Simple analysis of the definitions. ■ 

Lemma 4.2.6 T/ie function ^h is cr -additive on Ch, and /iff(0.ff) = 1- 

Proof. By definition hh^'h) = 1' nence it is sufficient to show u-additivity. Let q be an 
extended execution of M, and let be a set of incomparable extended executions of M such 
that C q = U g ' e eCg'- If q ends in 8, then contains only one element and u-additivity is 
trivially satisfied. Thus, assume that q does not end in 8, and hence q is a state of H , and that 
contains at least two elements. From Lemma 4.2.4, q is a prefix of each extended execution 
of 0. For each state q' of if, let Q q i be the set {q" £ | q' < q"}. We show u-additivity 
in two steps: first we assign an ordinal depth to some of the states of H and we show that q 
is assigned a depth; then we show that fJ,H(C q ) = J2 q 'e& l^ H (^-v) ^V ordinal induction on the 
depth assigned to q. 

The depth of each state q' within some cone C g » (q" < q'), where q" £ 0, is 0, and the depth 
of each state q' with no successors is 0. For each other state q' such that each of its successors 
has a depth, if {depth(q") \ 3 a (a,q") £ &5} has a maximum, then 

depth(q') = max({depth(q") \ 3 a (a,q") £ fi^}) + f, (4.13) 

otherwise, if {depth(q") \ 3 a (a,q") £ fi g /} does not have a maximum, then 

depth(q') = sup({depth(q") \ 3 a (a,q") £ fij}). (4.14) 

Consider a maximal assignment to the states of if, i.e., an assignment that cannot be extended 
using the rules above, and suppose by contradiction that q is not assigned a depth. Then 
consider the following sequence of states of H . Let go = Q, and, for each i > 0, let g 8 - be a state 
of H such that (a 8 ,g 8 ) £ 0, qi _ 1 , and g 8 - is not assigned a depth. For each i, the state g 8 - exists 
since otherwise, if there exists an i such that for each (a 8 ,g 8 ) £ 0, qi _ 1 , qi is assigned a depth, 
then g 8 _i would be assigned a depth. Note that the g 8 's form a chain under prefix ordering, i.e., 
for each i,j, if i < j then qi < qj. Consider the execution a^ = lim 8 - qj. From its definition, a^ 
is an execution of C q . Then, from hypothesis, a^ is an execution of U g ' e eCg' 5 an d therefore 
cioo is an execution of some C q i such that q' £ 0. By definition of a cone, q' is a prefix of a^,. 
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Thus, q' = qk for some k > 0. But then q^ is within the cone C q i, and thus it is assigned depth 
0. This contradicts the fact that q^ is not assigned any depth. 

Let 7 be the ordinal depth assigned to q. We show that fJ,H(C q ) = J2 q 'e& M-ff(CV) by 
ordinal induction on 7. If 7 = 0, then is either {q} or {qS}, and the result is trivial. Let 
7 be a successor ordinal or a limit ordinal. From Lemma 4.2.5, /iff(C g ) = J2( a g')efi M-ff (Cg') 
if 6 £ fl q , and fiH(C g ) = E(a, g ')efi q Mff(Cg') + m(C g s) if <5 £ O g . For each (a,q') G O g , 
C g / = U g // e e ;Cg"- Moreover, for each (a, g') £ O g , the depth of q' is less than 7. By induction, 
m(C q ') = E g "ee q , m(C q ")- Thus, mi(C q ) = E(a,g')efi q Eg» e e q , Mff(Cg") = Eg'eeMff(Cg') if 
<S £ O g , and n H (C q ) = E(a, g ')efi q E g »ee q , m(C q ") + m(C q s) = Eg'ee m(Cq') if ^ G O g . ■ 

Lemma 4.2.7 There exists a unique extension ii' H of ^h to F\(Ch)- 

Proof. There is a unique way to extend the measure of the cones to their complements since 
for each a, fJ,H(C a ) + I^h(^h — C a ) = 1. Therefore ii' H coincides with /i# on the cones and 
is defined to be 1 — fJ,H(C a ) f° r t ne complement of any cone C a . Since, by the countably 
branching structure of H , the complement of a cone is a countable union of cones, u-additivity 
is preserved. ■ 

Lemma 4.2.8 There exists a unique extension ^ of ii' H to ^(Cff)- 

Proof. The intersection of finitely many sets of F\(Ch) is a countable union of cones. Therefore 
(T-additivity enforces a unique measure on the new sets of F\(Ch)- ■ 

Lemma 4.2.9 There exists a unique extension ^ of ^ to F^(Ch)- 

Proof. There is a unique way of assigning a measure to the finite union of disjoint sets whose 
measure is known, i.e., adding up their measures. Since all the sets of F^(Ch) are countable 
unions of cones, u-additivity is preserved. ■ 

Theorem 4.2.10 There exists a unique extension P' H of ^h to the a-algebra T' H . 

Proof. By Theorem 3.1.2, define P' H to be the unique extension of ^ to T' H . ■ 

4.2.6 Finite Probabilistic Executions, Prefixes, Conditionals, and Suffixes 

We extend the notions of finiteness, prefix and suffix to the probabilistic framework. Here we 
add also a notion of conditional probabilistic execution which is not meaningful in the non- 
probabilistic case and which plays a crucial role in some of the proofs of Chapter 5. 

Finite Probabilistic Executions 

Informally, finiteness means that the tree representation of a probabilistic execution fragment 
has a finite depth. Thus, a probabilistic execution fragment H is finite iff there exists a natural 
number n such that the length of each state of H is at most n. 
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Figure 4-8: Examples of the prefix relation. 



Prefixes 



The idea of a prefix of a probabilistic execution fragment is more complicated than the definition 
of prefix for ordinary automata. To get a better understanding of the problem, consider the 
definition of prefix for ordinary execution fragments: a < a' iff either a = a', or a is finite and 
there is an execution fragment a" such that a' = a" a". Another way to interpret this definition 
is to observe that if a is finite, then there is exactly one point in a, which we call a point of 
extension, from which nothing is scheduled, and in that case a' is obtained by extending a from 
its unique point of extension. With the word "extending" we mean "adding transitions". In 
other words, an execution fragment a is a prefix of an execution fragment a' iff a' is obtained 
from a by adding transitions, possibly none, from all the points of extension of a, i.e., from 
all the points of a where nothing is scheduled. We apply the same observation to probabilistic 
execution fragments, where a point of extension is any point where 8 occurs. 

Example 4.2.6 (Prefixes) Consider the probabilistic execution fragment H of Figure 4-8. 
It is easy to see that si and «2 are points of extension in H . However, also so is a point 
of extension since in H nothing is scheduled from so with probability 1/2. The probabilistic 
execution fragment H' of Figure 4-8 is an extension of H . States si and «2 are extended with 
transitions labeled with c, and half of the extendible part of so is extended with the transition 
so — ► si, i.e., we have added the transition (so,ZY((a, si), 8)) to the extendible part of so- Since 
the extension from so overlaps with one of the edges leaving so in H, the effect that we observe 
in H' is that si is reached with a higher probability. 

Consider now the probabilistic execution fragment H" of Figure 4-8. H" is an extension 
of H', but this time something counterintuitive has happened; namely, the edge labeled with 
action c that leaves from state «2 has a lower probability in H" than in H' . The reason for this 
difference is that the extendible part of so is extended with a transition so — > «2 followed by 
«2 — ► s'. Thus, half of the transition leaving from «2 in H" is due to the previous behavior of 
H', and half of the transition leaving from «2 in H" is due to the extension from so- However, 
the probability of the cone C So b S2CS is the same in H' and in H". ■ 

A formal definition of a prefix works as follows. A probabilistic execution fragment if is a prefix 
of a probabilistic execution fragment H' , denoted by H < H', iff 

1. H and H' have the same start state, and 

2. for each state q of H, P H [C q ] < P H '[C q ]. 

Observe that the definition of a prefix for ordinary executions is a special case of the definition 
we have just given. 
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Figure 4-9: Conditionals and suffixes. 



Conditionals 



Let if be a probabilistic execution fragment of a probabilistic automaton M, and let q be either 
a state of if or a prefix of the start state of if . We want to identify the part of H that describes 
what happens conditional to the occurrence of q. The new structure, which we denote by H \q, 
is a new probabilistic execution fragment defined as follows: 

1. states(H\q) = {q 1 £ states(H) \ q < q'}; 

2. start(H\q) = min(states(H \q)) , where the minimum is taken under prefix ordering, 

3. for each state q' of if |g, tr , = tn . 

if | g is called a conditional probabilistic execution fragment. 

Example 4.2.7 (Conditionals) The probabilistic execution fragment ifi of Figure 4-9 is an 
example of a conditional probabilistic execution fragment. Specifically, ifi = H"\(soas2), where 
H" is represented in Figure 4-8. In Figure 4-9 we represent explicitly the states of ifi for clarity. 
The conditional operation essentially extracts the subtree of H" that starts with so«S2- ■ 

It is easy to check that (Q, H i , Tjji , P H i ) and (0#|C g , Tu\C q , i"ff|C g ) are the same probability 
space (cf. Section 3.1.8). Indeed, the sample sets are the same, the generators are the same, and 
the probability measures coincide on the generators. Thus, the following proposition, which is 
used in Chapter 5, is true. 

Proposition 4.2.11 Let H be a probabilistic execution fragment of a probabilistic automaton 
M, and let q be either a state of H , or a prefix of the start state of H . Then, for each subset 

e ofn H \ q , 
l. EeT Hlq iffEeTn. 

2. If E is an event, then Pr[E] = P H [C q ]P H \ q [E]. ■ 

Suffixes 

The definition of a suffix is similar to the definition of a conditional; the difference is that in 
the definition of H\>q we drop q from each state of H , i.e., we forget part of the past history. 
Formally, let if be a probabilistic execution fragment of a probabilistic automaton M, and let 
q be either a state of if or a prefix of the start state of H . Then H\>q is a new probabilistic 
execution fragment defined as follows: 

1. states(H>q) = {q'oq \ q' G states(H),q < q'}, 
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2. start(H>q) = min(states(H >q)) , where the minimum is taken under prefix ordering, 

3. for each state q' of H' tr , >q = tr H ^ ,>q. 

•1 1 q i q ^ q i 1 

H\>q is called a suffix of if. It is a simple inductive argument to show that H\>q is indeed 
a probabilistic execution fragment of M. Observe that the definition of a suffix for ordinary 
executions is a special case of the definition we have just given. 

Example 4.2.8 (Suffixes) The probabilistic execution fragment Hi of Figure 4-9 is an ex- 
ample of a suffix. Specifically, Hi = H"\>(soas2), where H" is represented in Figure 4-8. The 
suffixing operation essentially extracts the subtree of H" that starts with so«S2 an d removes 
from each state the prefix so«S2- ■ 

It is easy to check that the probability spaces Vu>q an d T^Hlq are i n a one-to-one correspondence 
through the measurable function / : &H>q - ► &H\q such that for each a £ &H>qi f( a ) = q^ ot. 
The inverse of / is also measurable and associates a\>q with each execution a of £ljj\q- Thus, 
directly from Proposition 4. 2. II, we get the following proposition. 

Proposition 4.2.12 Let H be a probabilistic execution fragment of a probabilistic automaton 
M , and let q be either a state of H , or a prefix of the start state of H . Then, for each subset 
E of £ln> q , 

1. EeT H>q iff(q~E)ef H . 

2. If E is an event, then Pn[q~ E] = PH[C q ]PH> q [E]. ■ 

4.2.7 Notation for Transitions 

In this section we extend the arrow notation for transitions that is used for ordinary automata. 
The extension that we present is meaningful for simple transitions only. 

An alternative representation for a simple transition (s, a, V) of a probabilistic automaton M 
is s — ► V . Thus, differently from the non-probabilistic case, a transition leads to a distribution 
over states. If V is a Dirac distribution, say V(s'), then we can represent the corresponding 
transition by s — ► s' . Thus, the notation for ordinary automata becomes a special case of the 
notation for probabilistic automata. If (s,a,V) is a simple combined transition of M, then we 
represent the transition alternatively by s — >c V ', where the letter C stands for "combined". 

The extension of weak transitions is more complicated. The expression s =>■ V means 
that V is reached from s through a sequence of transitions of M, some of which are internal. 
The main difference from the non-probabilistic case is that in the probabilistic framework the 
transitions involved form a tree rather than a linear chain. Formally, s =>■ V, where a is either 
an external action or the empty sequence and V is a probability distribution over states, iff 
there is a probabilistic execution fragment H such that 

1. the start state of H is s; 

2. Pfj[{a8 | aS £ 0^}] = 1, i.e., the probability of termination in H is 1; 

3. for each aS £ 0#, trace(a) = a; 
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Figure 4-10: A representation of a weak transition with action a. 
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Figure 4-11: A weak transition of a probabilistic automaton with cycles. 

4. V = lstate(8-strip(VH)), where 8-strip{Vjj) 1S the probability space V' such that 0' = 
{a | a 8 £ ^_ff}, and for each a £ 0', P'[a] = -PfffC^s]; 

5. for each state q of if, either £r;? is the pair (Istate(q) ,T>(8)) , or the transition that corre- 
sponds to £r;? i s a transition of M . 

A weak combined transition, s =^c P 5 is defined as a weak transition by dropping Condition 5. 
Throughout the thesis we also the extend the function 8- strip to extended execution fragment; 
its action is to remove the symbol 8 at the end of each extended execution fragment. 

Example 4.2.9 (Weak transitions) Figure 4-10 represents a weak transition with action 
a that leads to state si with probability 5/12 and to state S2 with probability 7/12. The 
action r represents any internal action. From the formal definition of a weak transition, a tree 
that represents a weak transition may have an infinite branching structure, i.e., it may have 
transitions that lead to countably many states, and may have some infinite paths; however, the 
set of infinite paths has probability 0. 

Figure 4-11 represents a weak transition of a probabilistic automaton with cycles in its 
transition relation. Specifically, H represents the weak transition so =>■ V, where P[so] = 1/8 
and P[s\] = 7/8. If we extend H indefinitely on its right, then we obtain a new probabilistic 
execution fragment that represents the weak transition so =>■ V(s\). Observe that the new 
probabilistic execution fragment has an infinite path that occurs with probability 0. Further- 
more, observe that there is no other way to reach state si with probability 1. ■ 

Remark 4.2.10 According to our definition, a weak transition can be obtained by concatenat- 
ing together infinitely many transitions of a probabilistic automaton. A reasonable objection 
to this definition is that sometimes scheduling infinitely many transitions is unfeasible. In the 
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timed framework this problem is even more important since it is feasible to assume that there 
is some limit to the number of transitions that can be scheduled in a finite time. Thus, a more 
reasonable and intuitive definition of a weak transition would require the probabilistic execution 
fragment H that represent a weak transition not to have any infinite path. All the results that 
we prove in this thesis are valid for the more general definition where H can have infinite paths 
as well as for the stricter definition where H does not have any infinite path. Therefore, we use 
the more general definition throughout. The reader is free to think of the simpler definition to 
get a better intuition of what happens. ■ 

An alternative way to represent a weak transition, which is used to prove the theorems of 
Chapter 8, is by means of a generator. If H represents a weak combined transition, then a 
generator can be seen as an object that chooses the combined transitions of M that lead to H 
(in Chapter 5 this object is also called an adversary). More precisely, a generator is a function 
O that associates a weak combined transition of M with each finite execution fragment of 
M. Before stating the formal properties that a generator satisfies, we give an example of the 
generator for the weak transition of Figure 4-10. 

Example 4.2.11 (Generators) Recall from Section 3.1.10 that U(x, y) denotes the probabil- 
ity space that assigns x and y probability 1/2 each. Then, the generator for the weak transition 
of Figure 4-10 is the function O where 

0(sTs[as' 3 ) = (s' 3 ,t,U(s 1 ,s 2 )) 
0(sts[) = (si,a,W(4,^)) OisTs 1 ^) = (s' 4 ,t,V(s 2 )) 
O(s) = ( S ,r,«K,4)) 0(sts> 2 ) = (s' 2 ,t,V(s' 5 )) 0(sts' 2 ts' 5 ) = (s' 5 ,aMsi,s 2 )) 

and 0(a) = (lstate(a),V(S)) for each a that is not considered above. The layout of the 
definition above reflects the shape of the probabilistic execution fragment of Figure 4-10. 

Thus, if we denote the probabilistic execution fragment of Figure 4-10 by H , O is the function 
that for each state q of H gives the combined transition of M that corresponds to tr^ . Function 
O is also minimal in the sense that it returns a transition different from (lstate(q),V(S)) only 
from those states q that are relevant for the construction of H . We call active all the states of 
H that enable some transition; we call reachable all the reachable states of H; we call terminal 
all the states q of H such that 6 G 0? . ■ 

Let M be a probabilistic automaton and let s be a state of M. A generator for a weak 

a \ ext(M) 

(combined) transition s =>■ V of M is a function O that associates a (combined) transition 
of M with each finite execution fragment of M such that the following conditions are satisfied. 

1. If O(a) = (s',V), then s' = Istate(a). Call a active if V ^ V(6). 

2. If abs' is active, then fstate(a) = s and (b,s') G $lo(a)- 

3. Call a reachable iff either a = s or a = a'bs' and (6, s') G ^lo(a')- Call a terminal iff a is 
reachable and Po(aas')[^] > 0. Then, for each terminal a, the trace of a is a \ ext(M). 

4. For each reachable execution fragment a = sa\Sia 2 s 2 ■ ■ -afc s fc 5 let 

P <* = 11 Po(sa 1 s 1 -a l s l )[{ a i + l s i + l)]-, 
0<i<k 
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Then, 

J7 = {Istate(a) \ terminal(a)} , 
and for each s' £ S7, 

Condition 1 says that the transition that 0(a) returns is a legal transition of M from Istate(a); 
Condition 2 guarantees that the active execution fragments are exactly those that are relevant 
for the weak transition denoted by O; Condition 3 ensures that the weak transition represented 
by O has action a \ ext(M); Condition 4 computes the probability space reached in the tran- 
sition represented by (9, which must coincide with V . The term P® represents the probability 
of performing a if O resolves the nondeterminism in M. Observe that terminal execution frag- 
ments must be reachable with probability 1 if we want the structure computed in Condition 4 
to be a probability space. 

Proposition 4.2.13 There is a weak combined transition s =>■ V of M iff there is a function 
O that satisfies the five conditions of the definition of a generator. 

Proof. Simple analysis of the definitions. ■ 

4.3 Parallel Composition 

In this section we extend to the probabilistic framework the parallel composition operator and 
the notion of a projection of ordinary automata. The parallel composition of simple probabilistic 
automata can be defined easily by enforcing synchronization on the common actions as in the 
non-probabilistic case; for general probabilistic automata, however, it is not clear how to give 
a synchronization rule. We discuss the problems involved at the end of the section. 

4.3.1 Parallel Composition of Simple Probabilistic Automata 

Two probabilistic automata M\ and Mi are compatible iff 

mf(Mi) n acts(M 2 ) = and acte(Mi) n int(M 2 ) = 0. 

The parallel composition of two compatible simple probabilistic automata Mi and M 2 , denoted 
by M1HM2, is the simple probabilistic automaton M such that 

1. states(M) = states{M\) X states(M 2 ). 

2. start(M) = start (M^ X start (M 2 ). 

3. sig(M) = (e^(Mi) U ext(M 2 ), mf(Mi) U int{M 2 )). 

4. ((si,s 2 ),a,P) G trans(M) iff V = V\ ® V 2 where 
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( S l,3 > S 2,0) 



( S l,l, S 2,o) ' d > (Sl,J,hl) 




( S l,0 > S 2,o) i"** ( s ;,2 ^ s 2,o) 




( S l,4 > S 2,o) 



( S l,5 > S 2,0J 



Figure 4-12: A probabilistic execution Iragment ol Mi||M2. 




Figure 4-13: The projection onto Mi of the probabilistic execution fragment of Figure 4-10. 



(a) if a G acts(M\) then (si,a,Vi) G trans(Mi), else V\ = V(s\), and 

(b) if a G acts(M2) then (s2,a,V2) G trans^M?), else P2 = ^( s 2)- 

Similar to the non-probabilistic case, two simple probabilistic automata synchronize on their 
common actions and evolve independently on the others. Whenever a synchronization occurs, 
the state that is reached is obtained by choosing a state independently for each of the proba- 
bilistic automata involved. 

4.3.2 Projection of Probabilistic Executions 

The Structure of the Problem 

Let M = M1HM2, and let if be a probabilistic execution fragment of M. We want to determine 
the view that Mi has of H , or, in other words, what probabilistic execution Mi performs in 
order for M1HM2 to produce H . To understand the complexity of the problem, consider the 
probabilistic execution fragment of Figure 4-12, and consider its projection onto Mi, represented 
in Figure 4-13. Actions a, b and c are actions of Mi, while action d is an action of M^. Thus, 
there is no communication between Mi and M^- Denote the probabilistic execution fragment 
of Figure 4-12 by H , and denote the probabilistic execution fragment of Figure 4-13 by H\. 
The projections of the states are ordinary projections of pairs onto their first component. The 
transitions, however, are harder to understand. We analyze them one by one. 

s lj0 The transition leaving s^o is obtained directly from the transition leaving (si,07 s 2,o) in 
H by projecting onto Mi the target states. 

s lj2 The transition leaving si^ is obtained by combining the transitions leaving states (si,2 5 s 2,o) 
and (si, 2,^2,1)5 each one with probability 1/2. The two transitions leaving (si,27 s 2,o) an d 
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( s i,2 5 s 2,i) have the same projection onto Mi, and thus the transition leaving si^ in Hi 
is s i,2 — ► s i,4- From the point of view of Mi, there is just a transition si^ — > £1,4; 
nothing is visible about the behavior of M^- 

To give a better idea of what we mean by "visible" , suppose that Mi is a student who 
has to write a report and suppose that the report can be written using a pen (action 
c) or using a pencil (action b). Suppose that the teacher may be able to get a pencil 
eraser (action d) and possibly erase the report written by the student once it is ready for 
grading. Then the scheduler is an arbiter who gives the student a pen if the teacher gets 
an eraser. If the student starts in state si^, then from the point of view of the student 
the material for the report is prepared (action a), and then the arbiter gives the student 
a pen with probability 1/2 and a pencil with probability 1/2; nothing is known about the 
time the the arbiter made the choice and the reason for which the choice was made. We 
can also think of the student as being alone in a room and the arbiter as being a person 
who brings to the student either a pen or a pencil once the material for the report is 
ready. 

The detailed computation of the transition leaving from si^ in Hi works as follows: we 
start from state (si, 2,^2,0)5 which is the first state reached in H where Mi is in si^, and 
we analyze its outgoing edges. We include directly all the edges labeled with actions of 
Mi in the transition leaving si^; for the other edges, we move to the states that they 
lead to, in our case (si,2, s 2,i) 5 an d we repeat the same procedure keeping in mind that 
the probability of the new edges must be multiplied by the probability of reaching the 
state under consideration. Thus, the edge labeled with a that leaves (si, 2,^2,0) is given 
probability 1/2 since its probability is 1/2, and the edge that leaves (si, 2,^2,1) is given 
probability 1/2 since the probability of reaching (si, 2,^2,1) from (si, 2,^2,0) is 1/2. 

For the transition leaving 5^4, we observe that in H there are two states, namely (si,4, £2,0) 
and (si, 4, £2,1)7 that can be reached separately and whose first component is 5^4. Each 
one of the two states is reached in H with probability 1/4. The difference between the 
case for state si^ and this case is that in the case for si^ state (si, 2,^2,0) occurs before 
(si,2, s 2,i) 5 while in this case there is no relationship between the occurrences of (si,4, .§2,0)7 
and (si,4,S2,i)- The transition leaving 5^4 depends on the state of M2 which, conditional 
on Mi being in 5^4, is 1/2 for 62,0 and 1/2 for 52,1- Thus, from the point of view of Mi, 
since the state of M2 is unknown, there is a transition from 5^4 that with probability 1/2 
leads to the occurrence of action b and with probability 1/2 leads to the occurrence of 
action c. Essentially we have normalized to 1 the probabilities of states («i, 4,^2,0) an d 
( s i,4 5 s 2,i) before considering their effect on Mi. 

The transition leaving s\ t i shows why we need the symbol 8 in the transitions of a proba- 
bilistic automaton. From state («i, 1,^2,0) there is a transition where action b occurs with 
probability 1/2 and action r occurs with probability 1/2. After r is performed, nothing 
is scheduled. Thus, from the point of view of Mi, nothing is scheduled from s\ t i with 
probability 1/2; the transition of M2 is not visible by Mi. 
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Action Restricted Transitions 

The formal definition of a projection relies on a new operation on transitions, called action 
restriction, which is used also in several other parts of the thesis. The action restriction op- 
eration allows us to consider only those edges of a transition that are labeled with actions 
from a designated set V. For example, V could be the set of actions of a specific probabilistic 
automaton. 

Formally, let M be a probabilistic automaton, V be a set of actions of M, and tr = (s,V) 
be a transition of M. The transition tr restricted to actions from V, denoted by tr \ V, is 
the pair (s,V) where V' is obtained from V by considering only the edges labeled with actions 
from V and by normalizing their probability to 1, i.e., 

. jy = I tt a ' s e ^ | a G F} if P[V] > 
1 {6} otherwise 

• if P[V] > 0, then for each (a,s') G fi', P'[(a,s')] = P[(a,s')]/P[V]. 

Two properties of action restriction concern commutativity with transition prefixing, and dis- 
tributivity with respect to combination of transitions. These properties are used in the proofs 
of other important results of this thesis. The reader may skip the formal statements for the 
moment and refer back to them when they are used. 

Proposition 4.3.1 For each q and tr such that one of the expressions below is defined, 

q~(tr \V) = (q~tr) \ V. 

Proof. Simple manipulation of the definitions. ■ 

Proposition 4.3.2 Let {t{\i^i be a collection of transitions leaving from a given state s, and 
let {pi}i e i be a collection of real numbers between and 1 such that J2ielPi — 1- Let V be a 
set of actions. Then 

(Tpitri) r v = y J iPtri[v ) (in \ v), 

where we use the convention that 0/0 = 0. 
Proof. Let 

(s,V) = Y^Pitrt, (4.15) 

i 

(s,V) = (J2 Pt t ri )\V, (4.16) 

i 

<'.*•> - £|giSr- fn (4 - l7) 

We need to show that V' and V" are the same probability space. 
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If P[V] = 0, then both V' and V" are V(6) and we are done. Otherwise, observe that 
neither 0' nor 0" contain <*). Consider any pair (a,s r ). Then, 

(a,s') G fi' 

iff (a, s') G fi and a G V from (4.16) and (4.15) 

iff 3 l (a,s') G Oi r ;,Pi > 0, and a G F from (4.15) 

iff 3i(a,s') G fiir-i fy an( i Pi > from the definition of tr\ \ V 

iff (a,s')en" l from (4.17). 

Consider now a pair (a, s') of 0'. From the definition of action restriction and (4.16), 

P'[(a,s')] = P[(a,s')]/P[V]. (4.18) 

From the definition of V (Equation (4.15)), the right side of Equation 4.18 can be rewritten 
into 

where ~%2j PiPt ri [V] is an alternative expression of P[V] that follows directly from (4.16). By 
multiplying and dividing each i summand of Expression 4.19 by Pir-J^]? we obtain 

T, ^p V L ( P *r i [(^')]/P tri [V]). (4-20) 

Since Ptr t [{ a i s ')]l Ptr t \Y] = Ptr t \v[( a i s ')]' fr° m t ne definition of V" (Equation (4.17)), Expres- 
sion 4.20 can be rewritten into P"[(a,s')]. Thus, P'[(a,s')] = P"[(a,s')]. This is enough to 
show that V = V". ■ 

Definition of Projection 

We give first the formal definition of a projection, and then we illustrate its critical parts by 
analyzing the example of Figures 4-12 and 4-13. It is very important to understand Expres- 
sions (4.21) and (4.22) since similar expressions will be used in several other parts of the thesis 
without any further explanation except for formal proofs. 

Let M = M1HM2, and let if be a probabilistic execution fragment of M. 

Let tr = (q, V) be an action restricted transition of H such that only actions of M 8 -, i = 1,2, 
appear in tr. Define the projection operator on the elements of S7 as follows: (a,q')\Mi = 
(a,q'\Mi), and 8\Mi = S. Recall from Section 3.1.5 that the projection can be extended 
to discrete probability spaces. The projection of tr onto Mi, denoted by tr\Mi, is the pair 
(q\Mi,P\Mi). 

The projection of H onto Mi, denoted by H\Mi, is the fully probabilistic automaton H' 
such that 

1. states(H') = {q\M % \ q G states(H)}; 

2. start(H') = {q\M % \ q G start(H)}; 

3. sig(H') = sig(Mi)- 
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4. for each state q of H', let q\H be the set of states of H that projected onto M 8 - give q, 
and let min(q\H) be the set of minimal states of q\H under prefix ordering. For each 
q' e (q]H), let 

pf = fHlC '| (4,21) 

The transition enabled from g in if' is 

tr ?' = E P^^Ms(M)](^ tacte(M t -))rM t -. (4.22) 



Each summand of Expression 4.22 corresponds to the analysis of one of the states of H that can 
influence the transition enabled from q in H' '. The subexpression (tr^, \ acts(Mi))\Mi selects 
the part of the transition leaving from q' where M 8 - is active, and projects onto M 8 - the target 
states of the selected part; the subexpression p q ) P*?[acts(Mi)] expresses the probability with 
which q' influences the transition enabled from q. P*J [acts( Mi)] is the probability that tr^, does 

"1 TT 

something visible by Mi, and pi is the probability of being in q' conditional on Mi being in q. 
Its value is given by Expression 4.21 and can be understood as follows. The state q' is either a 
minimal state of q] H or is reached from a minimal state through a sequence of edges with actions 
not in acts(Mi). The probability of being in q' , conditional on M 8 - being in q, is the normalized 
probability of being in the minimal state of q]H that precedes q' multiplied by the probability 
of reaching q' from that minimal state. We encourage the reader to apply Expression (4.22) to 
the states s^o, si,i, si,2, an d 5^4 of Figure 4-13 to familiarize with the definition. As examples, 
observe that min((si t obsi t 2)~\H) = {(si,o 5 s 2,o)^( s i,2 5 s 2,o)} an d that mm^s^o&si, 2^31,4)] if) = 
{(•§1,0, s 2 ,o)b(s 1}2 , s 2 ,o)a(s 1} 4, s 2 ,o), (si,o, s 2 ,o)b(s 1}2 , s 2 ,o)d(s 1}2 , s 2 ,i)a(s 1}4 , s 2 ,i)}. 

If we analyze the state 5^3 of Figure 4-13 and we use Expression 4.22 to compute the 
transition leaving 5^3, then we discover that the sum of the probabilities involved is not 1. 
This is because there is a part of the transition leaving (si,3,S2,o) where no action of Mi ever 
occurs. From the point of view of Mi nothing is scheduled; this is the reason of our choice of 
deadlock by default in the definition of the combination of transitions (cf. Section 4.2.2). 

We now move to Proposition 4.3.4, which is the equivalent of Proposition 3.2.1 for the 
probabilistic framework. Specifically, we show that the projection of a probabilistic execution 
fragment H of M1HM2 onto one of its components M 8 - is a probabilistic execution fragment 
of Mi. Proposition 3.2.1 is important because it shows that every computation of a parallel 
composition is the result of some computation of each of the components. One of the reasons 
for our use of randomized schedulers in the model is to make sure that Proposition 3.2.1 is 
valid. Before proving this result, we show that its converse does not hold, i.e., that there are 
structures that look like a probabilistic execution, that projected onto each component give a 
probabilistic execution of a component, but that are not probabilistic executions themselves. 

Example 4.3.1 (Failure of the converse of Proposition 4.3.4) Consider the probabilis- 
tic automata of Figure 4- 14. a, and consider a potential probabilistic execution of the composi- 
tion as represented in Figure 4-14. b. Denote the two probabilistic automata of Figure 4-14. a by 
Mi and M2, and denote the structure of Figure 4-14. b by H . The projections of if onto Mi and 
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a) Two compatible simple probabilistic automata. 




b) A potential probabilistic execution of the composition. 
Figure 4-14: A counterexample to the converse of the projection proposition. 

M2 give a probabilistic execution of Mi and M2, respectively. The diagrams of Figure 4-14. a 
can be viewed as the projections of H as well. However, H is not a probabilistic execution of 
M1HM2 since in no place of M\ it is possible to have a Dirac transition to s\ or S2- ■ 

The rest of this section is dedicated to the proof of the proposition that corresponds to Propo- 
sition 3.2.1 and to the proof of an additional result (Proposition 4.3.5) that gives a meaning to 
the denominator of Expression (4.21). We first state two preliminary properties of projection 
of transitions (Proposition 4.3.3). 

Proposition 4.3.3 Let M = M 1 \\M 2 . Then, for i = 1,2, 

2. (q~tr)\M i = (q\M i )~tr\M i . 
Proof. Simple manipulation of the definitions. ■ 

Proposition 4.3.4 Let M = M\\\M2, and let H be a probabilistic execution fragment of M . 
Then H\M\ £ prexec(Mi) and H\Mi £ prexec(Mi). 

Proof. We show that H\M\ £ prexec(Mi); the other statement follows from a symmetric 
argument. Let Hi denote H\M\. From Proposition 3.2.1, the states of H\ are execution 
fragments of M\. 

Consider now a state q of Hi. We need to show that there is a combined transition tr of 
M\ that corresponds to tr^ 1 , i.e., such that tr^ 1 = q^ tr. From Propositions 4.2.1 and 4.2.3, 
it is sufficient to show that for each state q' of q\H , there is a combined transition tr(q') of Mi 
such that 

(trf, \ acts{Mi))\Mi = q~ tr(q'). (4.23) 
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Then, the transition tr would be 

tr = E f} H PF[acts(Mi)]tr(q'). (4.24) 



J q i 

q'eq]H 



Proposition 4.2.1 is used to show that tr is a combined transition of Mi; Proposition 4.2.3 is 
used to show that q^ tr = tr} 1 . Since if is a probabilistic execution fragment of M, for each 
state q' of q\H there exists a combined transition tr'(q') of M such that 

tr} = q'~ tr'(q'). (4.25) 

From the definition of a combined transition, there is a collection of transitions {tr'(q',i)}i e i 
of M, and a collection of probabilities {pi}i£i, such that 

tr'(q') = J2p^'(q',i). (4.26) 

Note that each transition tr'(q', i) is a simple transition. From the definition of action restriction 
and (4.26), there is a subset J of/, and a collection of non-zero probabilities {p'j}j£j, such that 

tr'(q') \ acts(Mi) = ^<r'( g ',j). (4.27) 

3 

If we apply transition prefix with q' to both sides of Equation 4.27, we use commutativity 
of action restriction with respect to transition prefixing (Proposition 4.3.1) and (4.25) on the 
left expression, and we use distributivity of transition prefixing with respect to combination of 
transitions (Proposition 4.2.3) on the right expression, then we obtain 

tr} \ acts(M 1 ) = J^p'j (V '~ tr'(q',jj) . (4.28) 

3 

By projecting buth sides of (4.28) onto Mi, and using distributivity of projection with respect to 
combination of transitions (Proposition 4.3.3) and commutativity of projection and transition 
prefixing (Proposition 4.3.3) on the right expression, we obtain 



(tr} \ acts(M 1 ))\M 1 = J^p'j (<T (tr'(q' \j)\M 1 )) . (4.29) 

3 

From the distributivity of transition prefixing with respect to combination of transitions (Propo- 
sition 4.2.3), Equation 4.29 becomes 

(tr} \ acts(M 1 ))\M 1 = q~Y t p , j (tr'(q , ,j)\M 1 ). (4.30) 

3 

From standard properties of the projection of product probability distributions (cf. Sec- 
tion 3.1.6) and the definition of parallel composition, each tr'(q',j)\M\ is a transition of Mi. 
Thus, ^2jP'jtr'(q',j)\Mi is the combined transition of Mi that satisfies Equation 4.23. 

Finally, we need to show that each state q of Hi is reachable. This is shown by induction 
on the length of q, where the base case is the start state of Hi. The start state of Hi is 
trivially reachable. Consider a state qas of Hi. By induction, q is reachable. Let q' be a 
minimal state of (qas)~\H. Then, q' = q"a(s,S2), where q" is a state of q\H and «2 is a state 
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of M 2 . Moreover, (a,q r ) G Sl tr H , and thus, (a, qas) G Qt ir H |' aC i S (Mi))fMi- Since no edges with 

probability are allowed in a probabilistic automaton, the 
and thus (a, qas) G Q q . This means that qas is reachable. 



q" "■ q" 

probability are allowed in a probabilistic automaton, the term p q ), P*f,[acts(Mi)] is not 0, 



We conclude this section with another property of projections that gives a meaning to the 
denominator of Expression (4.21). Specifically, the proposition below allows us to compute the 
probability of a finitely satisfiable event of the projection of a probabilistic execution fragment 
H by computing the probability of a finitely satisfiable event of H . Observe that the right 
expression of (4.31) is indeed the denominator of (4.21). 

Proposition 4.3.5 Let M = M\\\M2, and let H be a probabilistic execution fragment of M . 
Let Hi be H\Mi, i = 1,2. Let q be a state of Hi. Then, 

P Ht [C q ]= E p H[C q >]. (4.31) 

q' (zmin(q~\H) 

Proof. The proof is by induction on the length of q, where the base case is for the start state 
of Hi. If q is the start state of Hi, then the start state of H is the only minimal state of q]H . 
Both the cones denoted by the two states have probability 1. 

Consider now the case for qas. From the definition of the probability of a cone, 

P Hl [C qas ] = P Hl [C q ]P q Hl [(a, qas)]. (4.32) 

By using Expression 4.22 and the definitions of action restriction and projection, the term 
P^ 1 [(a, qas)] can be rewritten into 

E p q q ]H P?[acts(M t )]{ J2 P^[(a,q")]/P^[acts(M t )]\, (4.33) 

q'eq]H \ q "e(qas)]H\(a, q ")enf, 

which becomes 



E Pi' \ E P?[(a,g")]\, (4.34) 

q'eq]H \ q "e(qas)]H\(a, q ")en H , 



q' 



after simplifying the term P*?[acts(Mi)]. The case when P*?[acts(Mi)] = is not a problem 

"1 TT 

since the innermost sum of Expression 4.33 would be empty. By expanding p ) in Expres- 
sion 4.34 with its definition (Equation 4.21), applying induction to Pfj^Cg] in Expression 4.32, 
and simplifying algebraically, Equation 4.32 can be rewritten into 

P Hl [C q as]= E E P H [C q >]P^[(a,q")]. (4.35) 

q'eq]H q"e(qas)]H\(a,q")enX 

1 TT 

Indeed, the denominator of the expansion of p J coincides with the expansion of PH 1 [C q ]. 

From the definition of the probability of a cone, the terms Pfj[C q i]P^[(a,q")] that appear 
in Equation 4.35 can be rewritten into Pff[C g »]. 
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Consider now one of the states q" of the right side of Equation 4.35. Then q"\Mi = qas, and 
there exists a state q' of q~\H such that (a,q") G O g '. This means that q" can be expressed as 
q'as' for some state s' of M. Since g' [M 8 - = g, then g" is a minimal state of (qas)~\H . Conversely, 
let q" be a minimal state of (gas)] if. Then q" can be expressed as q'as' for some state q' of i? 
and some state s' of M (otherwise q" would not be minimal). Moreover, q' is a state of q\H 
and (a, g") G Oi/. Thus, g" is considered in Equation 4.35. Finally, each minimal state q" of 
(qas)~\H is considered at most once in Equation 4.35, since there is at most one state q' in H 
such that (a, q") G Oi/. Thus, Equation 4.35 can be rewritten into 

^[CW]= E ^KV'L ( 4 -36) 

q" (zmin((qas)~\H) 

which is what we needed to show. ■ 

4.3.3 Parallel Composition for General Probabilistic Automata 

In this section we give an idea of the problems that arise in defining parallel composition for 
general probabilistic automata. The discussion is rather informal: we want to give just an idea 
of why our intuition does not work in this case. 

The main problem that needs to be addressed is to choose when two transitions should 
synchronize and how the synchronization would occur. We analyze the problem through some 
toy examples. Consider two probabilistic automata M\,Mi with no internal actions and such 
that ext(M\) = {a,b,c,d} and ext(M2) = {a, 6, c,e}. Let (si, S2) be a reachable state of M1HM2, 
and consider the following cases. 

1. Suppose that from state si of M\ there is a transition tr\ giving actions a, b probability 
1/2 to occur, and suppose that from state S2 of Mi there is a transition tr^ giving actions 
a, b probability 1/2 to occur. 





If we choose not to synchronize tr\ and tr^, then the only transitions that can be syn- 
chronized are the simple transitions, leading to a trivial parallel composition operator 
that does not handle any kind of transition with probabilistic choices over actions. The 
transitions tr\ and tr^ cannot be scheduled even independently, since otherwise the CSP 
synchronization style would be violated. 

If we choose to synchronize tr\ and tr^, then both M\ and Mi choose an action between 
a and b. If the actions coincide, then there is a synchronization, otherwise we have two 
possible choices in our definition: either the system deadlocks, or the random draws are 
repeated. The first approach coincides with viewing each probabilistic automaton as de- 
ciding its next action probabilistically independently of the other interacting automaton; 
the second approach is the one outlined in [GSST90], where essentially deadlock is not 
allowed, and assumes some dependence between the involved probabilistic automata. 

For the rest of the discussion we assume that the transitions tr\ and tr^ do synchronize; 
however, we leave unspecified the way in which tr\ and tr^ synchronize. 
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2. Suppose that from state si of Mi there is a transition tr\ giving actions a, b probability 
1/2 to occur, and suppose that from state S2 of Mi there is a transition tr^ giving actions 
a,c probability 1/2 to occur. 





tr 2 : s 2 



Note that actions a, b and c are all in common between Mi and M^- If we choose not 
to synchronize tr\ and tr^, then only transitions involving the same sets of actions can 
synchronize. However, we have the same problem outlined in Case 1, where neither tri, 
nor tr-2 can be scheduled independently. 

If we choose to synchronize tr\ and tr^, then, since a is the only action that is in common 
between tr\ and tr^, the only action that can occur is a. Its probability is either 1 or 1/4 
depending on how the synchronization in Case 1 is resolved. However, in both cases the 
only action that appears in the sample space of the composite transition is a. 

For the rest of the discussion we assume that the transitions tr\ and tr^ do synchronize. 
Once again, we leave unspecified the way in which tr\ and tr^ synchronize. 

3. Suppose that from state si of Mi there is a transition tr\ giving actions a, 6, d probability 
1/3 to occur, and suppose that from state «2 of Mi there is a transition tr^ giving actions 
a, 6, e probability 1/3 to occur. 





In this case each transition has some actions that are in common between Mi and M2, 
and some actions that are not in common. 

If we choose not to synchronize tr\ and tr^, then, beside the fact that tr\ and tr^ could not 
be scheduled independently, the parallel composition operator would not be associative. 
Consider two new probabilistic automata M{,M2 with the same actions as Mi and M2, 
respectively. Suppose that from state s' x of M[ there is a transition tr[ giving actions a, b 
probability 1/2 to occur, and suppose that from state s 2 of M' 2 there is a transition tr' 2 
giving actions a, b probability 1/2 to occur. 



"•;•• 




"r 




If we consider (M{||Mi)||(M2||M2), then in state ((s^, si), (S2, s 2 )) ^ r i w °uld synchronize 
with tr\ leading to a transition that involves actions a and b only, tr^ would synchronize 
with tr' 2 leading to a transition that involves actions a and b only, and the two new 
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transitions would synchronize because of Case 1, leading to a transition that involves 
actions a and b. If we consider (M-[||(Mi||M 2 ))||M 2 , then in state ((s^, (si, s 2 )), s 2 ) ^ r i 
and tr-2 would not synchronize, and thus associativity is broken. 

If we choose to synchronize tr\ and £r 2 , then problems arise due to the presence of actions 
that are not in common between M\ and M 2 . In particular we do not know what to do if 
M\ draws action d and M 2 draws action e, or if Mi draws action d and M 2 draws action 
a. Since we do not want to assume anything about the respective probabilistic behaviors 
of Mi and M 2 , at least the first case is an evident case of nondeterminism. 

However, even by dealing with the first case above by means of nondeterminism, only 
one of actions d, e can be performed. Suppose that d is chosen, and thus Mi performs a 
transition while M 2 does not. What happens to M 2 ? Is action e supposed to be chosen 
already after d is performed? Otherwise, what is the probability for e to occur? At this 
point we do not see any choice that would coincide with any reasonable intuition about 
the involved systems. 

In the second case we are sure that action a cannot occur. Does this mean that action d 
occurs for sure? Or does this mean that a deadlock can occur? With what probabilities? 
Once again, intuition does not help in this case. 

The main problem, which is evident especially from Case 3, is that we do not know who is in 
control of a system, and thus, whenever there is a conflict that is not solved by nondeterminism 
alone, we do not know what probability distribution to use to resolve the conflict. However, 
if we decorate probabilistic automata with some additional structure that clarifies who is in 
control of what actions [LT87], then parallel composition can be extended safely to some forms 
of general probabilistic automata, where the external actions are partitioned into input and 
output actions, the transitions that contain some input action are simple transitions, and input 
actions are enabled from every state (cf. Section 13.2.2). An observation along this line appears 
in [WSS94]. 

4.4 Other Useful Operators 

There are two other operators on probabilistic automata that should be mentioned, since they 
are used in general on ordinary automata. In this section we provide a short description of 
those operators. Since the relative theory is simple, this is the only point where we mention 
these operators during the development of the probabilistic model. 

4.4.1 Action Renaming 

Let p be a one-to-one function whose domain is acts(M). Define Rename P (M) to be the 
probabilistic automaton M' such that 

1. states(M') = states(M). 

2. start(M') = start(M). 

3. sig(M') = (p(ext(Mj),p(int(M))). 
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4. (s,V) G trans(M') iff there exists a transition (s,V') of M such that 7-* = p'(V'), where 
p'((a,s')) = (p(a),s r ) for each (a,s') G 0', and p'(#) = <*>. 

Thus, the effect of Rename p is to change the action names of M. The restriction on p to be 
one-to-one can be relaxed as long as internal and external actions are not mixed, i.e., there is 
no pair of actions a, b where a is an external action, b is an internal action, and p(a) = p(b). 

4.4.2 Action Hiding 

Let M be a probabilistic automaton, and let /be a set of actions. Then Hidei(M) is defined 
to be a probabilistic automaton M' that is the same as M, except that 

sig(M') = (ext(M) - I, int(M) U I). 

That is, the actions in the set / are hidden from the external environment. 

4.5 Discussion 

The generative model of probabilistic processes of van Glabbeek et al. [GSST90] is a special 
case of a fully probabilistic automaton; simple probabilistic automata are partially captured 
by the reactive model of [GSST90] in the sense that the reactive model assumes some form 
of nondeterminism between different actions. However, the reactive model does not allow 
nondeterministic choices between transitions involving the same action. By restricting simple 
probabilistic automata to have finitely many states, we obtain objects with a structure similar to 
that of the Concurrent Labeled Markov Chains of [Han91]; however, in our model we do not need 
to distinguish between nondeterministic and probabilistic states. In our model nondeterminism 
is obtained by means of the structure of the transition relation. This allows us to retain most 
of the traditional notation that is used for automata. 

Our parallel composition operator is defined only for simple probabilistic automata, and thus 
a natural objection is that after all we are dealing just with the reactive model. Furthermore, 
the reactive model is the least general according to [GSST90]. Although we recognize that our 
simple probabilistic automata constitute a restricted model and that it would be desirable to 
extend the parallel composition operator to general probabilistic automata, we do not think that 
it is possible to use the classification of [GSST90] to judge the expressivity of simple probabilistic 
automata. The classification of [GSST90] is based on a synchronous parallel composition, while 
our parallel composition is based on a conservative extension of the parallel composition of CSP 
[Hoa85]. Furthermore, in the classification of [GSST90] a model is more general if it contains 
less nondeterminism, while in our model nondeterminism is one of the key features. 
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Chapter 5 

Direct Verication Stating a 
Property 



This chapter presents a method to study the properties that a probabilistic automaton satisfies. 
We describe how an informally stated property can be made rigorous, and we show how simple 
statements can be combined together to give more complex statements. In Chapter 6 we develop 
techniques to prove from scratch that a probabilistic automaton satisfies a given property. 

Part of this chapter is based on discussion with Isaac Saias who provided us with the 
motivations for the definition of progress statements (Section 5.5) and for the statement of the 
concatenation theorem (Theorem 5.5.2). 

5.1 The Method of Analysis 

If we read through the papers on randomized algorithms and we look at the statements of 
correctness, we see claims like 

" Whenever the algorithm X starts in a condition Y , no matter what the adversary 
does, the algorithm X achieves the goal Z with probability at least p." 

For convenience, denote the statement above by S. A possible concrete instantiation of S is 
the following: 

" Consider a distributed system X , composed of n processors, that provides services 
under request and suppose that some request R comes. Then, independently of the 
relative order in which the n processors complete their operations (no matter what 
the adversary does), a response to R is given eventually (the goal Z ) with probability 
at least 2/3. 

Let us try to understand the meaning of the statement S . First of all, in S there is an entity, 
called adversary, that affects the performance of algorithm X . The adversary is seen as a 
malicious entity that degrades the performance of X as much as possible. 

If X is a distributed algorithm that runs on n separate processes, then the adversary is the 
entity that chooses what process performs the next transition, and possibly what the external 
environment does. To account for all the possible ways to schedule processes, the adversary 
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Figure 5-1: A toy resource allocation protocol. 

bases its choices on a complete knowledge of the state of a system, including its past history. If 
the algorithm is represented as a probabilistic automaton, then an adversary is the object that 
resolves the nondeterminism. In other words, an adversary is a scheduler seen as a malicious 
entity. 

However, not all the schedulers guarantee in general that some specific property is satisfied. 
For example, an adversary is usually required to be fair to all the processes of a system in 
order to guarantee progress. In other cases, an adversary is not allowed to base its choices on a 
complete knowledge of the history of a system: the correctness of an algorithm may rely on the 
adversary not to use the results of previous random draws in choosing the next process to be 
scheduled. Thus, in the statement S there is usually an implicit assumption that an adversary 
has some limitations. 

Example 5.1.1 (A toy resource allocation protocol) Figure 5-1 illustrates a toy scenario 
where correctness is guaranteed only for adversaries that do not know the outcome of the random 
draws of the processes. Two processes Mi and Mi compete for two resources R\ and Ri- Each 
process continuously runs through the following cycle: 

1. flip a coin to choose a resource; 

2. if the chosen resource is free, then get it; 

3. if you hold the resource, then return it. 

That is, each process continuously tries to get a randomly chosen resource and then returns it, 
possibly after using the resource. Of course this is a stupid protocol, but it highlights several 
aspects of randomized distributed algorithms. Suppose every adversary to be fair, meaning 
that both processes perform infinitely many transitions. A malicious adversary can create a 
situation where M\ never succeeds in obtaining a resource with an arbitrarily high probability. 
The adversary works as follows. Fix an arbitrary probability p such that < p < 1, and consider 
a collection of probabilities {pi}i£N such that YliPi = P- We know that such a collection 
of probabilities exists. Then the adversary works in rounds, where at round i the following 
happens: 

a. Mi is scheduled until it flips its coin; 

b. M2 is scheduled for sufficiently many times so that it gets the resource chosen by Mi 
with probability at least pt (finitely many times are sufficient). As soon as M2 gets the 
resource chosen by Mi the control goes to c; 
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c. Mi is scheduled to check its resource and fails to get it. 

In this case Mi fails to obtain a resource with probability at least p. On the other hand, if 
an adversary is not allowed to base its choices on the outcome of the coin flips, or better, 
if an adversary chooses the next process that performs a transition based only on the order 
in which processes were scheduled in the past, then each process eventually gets a resource 
with probability 1 (this fact is proved in Section 6.6). Such an adversary is called an oblivious 
adversary or an off-line scheduler. ■ 

Let us move back to the problem of understanding the statement S. Consider a valid adversary 
A, i.e., an adversary that satisfies the limitations that are implicitly assumed for S. Let M 
be a probabilistic automaton that describes algorithm X , and consider an arbitrary starting 
point q for M, i.e., q is a finite execution fragment of M that describes a partial evolution of 
M. If we let A resolve the nondeterminism in M starting from the knowledge that q occurred, 
then we obtain a probabilistic execution fragment of M, which we denote by prexec(M,A,q). 
According to S, if q satisfies condition Y, then prexec(M,A,q) should satisfy property Z with 
probability at least p. However, Z is a property of M, and not a property of prexec(M, A, q). 
Thus, we need a way to associate with prexec(M,A,q) the event that expresses Z. The object 
that does this operation is called an event schema. At this point it is possible to formalize S 
by stating the following: 

LL For each valid adversary A and each valid starting condition q, the probability of 
the event associated with prexec(M, A, q) is at least p." 

This is an example of what we call a probabilistic statement. 

A probabilistic statement that plays an important role in our analysis is denoted by 

U -^Advs U', (5.1) 

where U and U' are sets of states, p is a probability, and Advs is a set of adversaries. We call 
such a statement a progress statement. Its meaning is that if a protocol starts from a state of 
U, then, no matter what adversary of Advs is used to resolve the nondeterminism, some state of 
U' is reached with probability at least p. A progress statement is a probabilistic generalization 
of the leads-to operator of UNITY [CM88]. 

Example 5.1.2 It is possible to show (cf. Section 6.6) that the toy resource allocation protocol 
satisfies 1Z — >Advs -Mi, where 1Z is the set of reachable states of Mi||M2, M\\s the set of states 

1/2 

of Mi||M2 where Mi holds a resource, and Advs is the set of fair oblivious and adversaries for 
M1HM2, i.e., the set of adversaries that are fair to each process and that do not base their 
choices on the outcomes of the coin flips (cf. Example 5.6.2 for a formal definition of a fair 
oblivious adversary). ■ 

Progress statements are important because, under some general conditions, they can be com- 
bined together to obtain more complex progress statements, thus allowing the decomposition 
of a complex problem into simpler problems. 
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Example 5.1.3 Suppose that in some system M whenever a request is pending (M is in a 
state of some set V, a token is given eventually with probability at least 1/2 (reaching a state 
of some set T), and suppose that whenever a token is given a response is given eventually with 
probability at least 1/3 (reaching a state of some set Q). That is, 

V —+Advs T and T —*Advs Q- (5.2) 

1/2 1/3 V ' 

Then, it is reasonable to conclude that whenever a request is pending a response is given 
eventually with probability at least 1/6, i.e., 

V —>Advs Q- (5.3) 

1/2 V ' 

This result is a consequence of the concatenation theorem (cf. Theorem 5.5.2). ■ 

Example 5.1.4 Consider the toy resource allocation protocol again. We know from Exam- 
ple 5.1.2 that 

n --+Advs Ml (5.4) 

1/2 

It is also possible to show that 

U => U Unless M u (5.5) 

where 1Z =>■ 1Z Unless M\ is a UNITY [CM88] expression stating that whenever a system is in a 
state of 1Z the system remains in a state of 1Z unless a state of .Mi is reached. This means that 
(5.4) is applicable from any point in the evolution of the toy resource allocation protocol, and 
this fact, together with the condition that every adversary is fair, is succicient to guarrantee 
that 

n -^Advs M x (5.6) 

(cf. Proposition 5.5.6). The reader familiar with UNITY may note that the combination of 
(5.4) and (5.5) is a probabilistic generalization of the ensures operator of Chandy and Misra 

[CM88]. ■ 

To see more significative applications of progress statements the reader is referred to Chapter 6, 
where we prove the correctness of the randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81], and we prove the correctness of the randomized algorithm of Ben- Or for 
agreement in asynchronous networks in the presence of stopping faults [B083]. Instead, the final 
part of this chapter concentrates on standard methods to specify event schemas and adversary 
schemas, and on the relationship between deterministic and general (randomized) adversaries. 
The main lesson that we learn is that for a large class of probabilistic statements it is possible 
to prove their validity by considering only deterministic adversaries, i.e., adversaries that do 
not use randomization in their choices. The reader who is reading only the first section of each 
chapter should move to Chapter 6 at this point and skip the rest of this section. 

We said already that an event schema is a rule to associate an event with each probabilistic 
execution fragment. More formally, an event schema is a function that given a probabilistic 
execution fragment H returns an event of Tu- However, we have not given any method to 
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specify an event schema. Our definition of an event schema is very general since it allows for 
any kind of rule to be used in determining the event associated with a probabilistic execution 
fragment. On the other hand, there is a specific rule which is used in most of the existing 
literature on randomized algorithms. Namely, given a probabilistic automaton M, a set of 
execution fragments of M is fixed, and then, given a probabilistic execution fragment H of M, 
the event associated with if is fl 0#. We call such an event schema an execution-based event 
schema. Since the start state of a probabilistic execution fragment contains part of the history of 
M, and since in general we are interested in what happens only after the probabilistic execution 
fragment starts, we refine the definition of an execution-based event schema by associating a 
probabilistic execution fragment H with the event n (O^-o^o ), where q^ is the start state of 
H . In this way a progress statement can be stated in terms of execution-based event schemas, 
where is the set of execution fragments of M that contain at least one occurrence of a state 
from U' . 

To specify an adversary schema there are two main restrictions that are usually imposed. 
One possibility is to restrict the kind of choices that an adversary can make, and the other 
possibility is to restrict the on-line information that an adversary can use in making its choices. 
The first kind of restriction is usually achieved by fixing a set of execution fragments before- 
hand and requiring that all the probabilistic execution fragments H generated by an adversary 
satisfy Q,jj C 0. We call the corresponding adversary schema an execution-based adversary 
schema. The second kind of restriction is achieved by imposing a correlation on the choices of 
an adversary on different inputs. We call the corresponding adversary schema an adversaries 
schema with partial on-line information. 

Example 5.1.5 An example of an execution-based adversary schema is the set of fair adver- 
saries for n processes running in parallel. In this case is the set of execution fragments of 
the composite system where each process performs infinitely many transitions. An example of 
an adversary schema with partial on-line information is the set of oblivious adversaries for the 
toy resource allocation protocol. Execution-based adversary schemas and adversary schemas 
with partial on-line information can be combined together. An example of an execution-based 
adversary schema with partial on-line information is the set of fair and oblivious adversaries 
for the toy resource protocol (cf. Example 5.6.2). ■ 

Exacution-based adversaries and event schemas give us a good basis to study the relationship 
between deterministic and general adversaries. Roughly speaking, and adversary is determin- 
istic if it does not use randomness in its choices. Then the question is the following: "does 
randomness add power to an adversary?" The answer in general is "yes"; however, there are 
several situations of practical relevance where randomness does not add any power to an ad- 
versary. In particular, we show that randomization does not add any power when dealing with 
finitely satisfiable execution-based event schemas in two scenarios: execution-based adversary 
schemas and adversary schemas with partial on-line information. 

5.2 Adversaries and Adversary Schemas 

An adversary , also called a scheduler, for a probabilistic automaton M is a function A that 
takes a finite execution fragment a of M and returns a combined transition of M that leaves 
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from Istate(a). Formally, 

A :frag*(M) -+ Probs(ctrans(Mj) 

such that if A(a) = (s,V), then s = Istate(a). 

An adversary is deterministic if it returns either transitions of M or pairs of the form 
(s,V(8)), i.e., the next transition is chosen deterministically. Denote the set of adversaries 
and deterministic adversaries for a probabilistic automaton M by Advs(M) and DAdvs(M), 
respectively. We introduce deterministic adversaries explicitly because most of the existing 
randomized algorithms are analized against deterministic adversaries. In Section 5.7 we study 
the connections between deterministic adversaries and general adversaries. 

As we have noted already, the correctness of an algorithm may be based on some specific 
assumptions on the scheduling policy that is used. Thus, in general, we are interested only in 
some of the adversaries of Advs(M). We call a subset of Advs(M) an adversary schema, and 
we use Advs to denote a generic adversary schema. Section 5.6 describes in more detail possible 
ways to specify an adversary schema. 

5.2.1 Application of an Adversary to a Finite Execution Fragment 

The interaction of an adversary A with a probabilistic automaton M leads to a probabilistic 
execution fragment, where the transition enabled from each state is the transition chosen by 
A. Given a finite execution fragment a of M, the probabilistic execution of M under A with 
starting condition a, denoted by prexec(M, A, a), is the unique probabilistic execution fragment 
H of M such that 

1. start(H) = {a}, and 

2. for each state q of if, the transition tr^ is q ~ A(q). 

Condition 2 ensures that the transition enabled from every state q of H is the transition chosen 
by A. It is a simple inductive argument to show that H is well defined. 

5.2.2 Application of an Adversary to a Finite Probabilistic Execution Frag- 
ment 

From the theoretical point of view, we can generalize the idea of the interaction between an 
adversary and a probabilistic automaton by assuming that the start condition is a finite prob- 
abilistic execution fragment of M. In this case the adversary works from all the points of 
extension of the starting condition. The resulting probabilistic execution fragment should be 
an extension of the starting condition. Formally, if if is a finite probabilistic execution fragment 
of M, then the probabilistic execution of M under A with starting condition if, denoted by 
prexec(M, A, H), is the unique probabilistic execution fragment H' of M such that 

1. start(H') = start (H), and 

2. for each state q of if', if q is a state of H , then tr^ is 

Vf \acts(HJ)+(l-p)(q~A(qj), 
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Figure 5-2: An example of the action of an adversary on a probabilistic execution fragment, 
where 
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and if q is not a state of H , then tr^ is q ~ .4(g). 
Once again, it is a simple inductive argument to show that H' is well defined. 

Example 5.2.1 (Extension of a finite probabilistic execution fragment) Before prov- 
ing that H' is an extension of H , we describe in more detail how the definition above works. 
The difficult case is for those states q of H' that are also states of H . Consider the example of 
Figure 5-2. Let A choose go — ► Q on input go, choose q — ► qi on input g, and choose 8 on all 
other inputs. The probabilistic execution fragment H' of Figure 5-2 is the result of the action 
of A on the probabilistic execution fragment H of Figure 5-2. In H' there are two ways to reach 
q: one way is by means of transitions of H , and the other way is by means of transitions due 
to A that originate from go- Thus, a fraction of the probability of reaching q in H' is due to 
H , while another fraction is due to the effect of A on H . The weight with which the transition 
tr^ is considered in H' is the first fraction of the probability of reaching q, which is expressed 
by PhICq]/ PH'[Cq\. In our example the fraction is 1/2. However, in our example the transition 
tr^ may also leads to 8 with probability 1/2, and the part of tr^ that leads to 8 should be 
handled by A. For this reason in the left term of the definition of tr„ we discard 8 from tr„ 
and we add a multiplicative factor P^[acts(H)] to the weight. Thus, in our example, three 
quarters of the transition leaving from q in H' are controlled by A. Note that the probability 
of reaching q\ from go is the same in H and H'. ■ 

Proposition 5.2.1 Let M be a probabilistic automaton, and let A be an adversary for M . 
Then, for each finite probabilistic execution fragment H of M , the probabilistic execution frag- 
ment generated by A from H is an extension of H , i.e., 

H < prexec(M,A,H). 
Proof. Denote prexec(M,A,H) by H' . We need to prove that for each state g of H, 

P H [Cq] < P H >[Cq}. (5.7) 

If g is the start state of H , then g is also the start state of H 1 ', and (5.7) is satisfied trivially. 

Consider now a state qas of H that is not the start state of H . Then g is a state of H . 
From the definition of the probability of a cone, 

PwiCqas] = Pw[C q ]Pf[(a, qas)]. (5.8) 
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From the definition of tr q , 

Pf[(a,qas)} = J^f? [(«, qas)] + (l - ^^lf[acts(H)]\ P A{q) [(a,qas)]. (5.9) 

Here we have also simplified the expression P^[acts(H)] in the first term as we did in the proof 
of Proposition 4.3.5 (Expressions (4.33) and (4.34)). We will not mention this simplification 
any more in the thesis. 

If we remove the second term from the right expression of Equation (5.9), turning Equa- 
tion (5.9) into an inequality, we obtain 



-,11' u „m ^ p H[C q \ vH 

P H '[C q 



P q H [(a, qas)} > " L ^ gJ P g [(a, qas)}. (5.10) 



By using (5.10) in (5.8), and simplifying the factor Pfj'[C q }, we obtain 

P H '[C qas } > P H [C q }P q H [(a, qas)}. (5.11) 

The right part of (5.11) is Pff[C gas ]. Thus, we conclude 

P H '[C qa s] > P H [C qas }. (5.12) 



5.3 Event Schemas 

In the informal description of a probabilistic statement we said that we need a rule to associate 
an event with each probabilistic execution fragment. This is the purpose of an event schema. 
An event schema for a probabilistic automaton M, denoted by e, is a function that associates an 
event of Tu with each probabilistic execution fragment if of M. An event schema e is finitely 
satisfiable iff for each probabilistic execution fragment H the event e(H) is finitely satisfiable. 
Union, intersection and complementation of event schemas are defined pointwise. Similarly, 
conditional event schemas are defined pointwise. 

The best way to think of an event schema is just as a rule to associate an event with 
each probabilistic execution fragment. Although in most of the practical cases the rule can be 
specified by a set of executions (cf. Section 5.3.2), part of our results do not depend on the 
actual rule, and thus they would hold even if for some reason in the future we need to study 
different rules. Moreover, event schemas allow us to simplify the notation all over. 

5.3.1 Concatenation of Event Schemas 

If e is a finitely satisfiable event schema, i.e., for each probabilistic execution fragment H the 
event e(H) can be expressed as a union of cones, then it means that in every execution of e(H) 
it is possible to identify a finite point where the property denoted by e is satisfied. Sometimes 
we may be interested in checking whether a different property, expressed by another event 
schema, is satisfied eventually once the property expressed by e is satisfied. That is, we want 
to concatenate two event schemas. 
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Formally, let e\ , e 2 be two event schemas for a probabilistic automaton M where e\ is finitely 
satisfiable, and let Cones be a function that associates a set Cones(H) with each probabilistic 
execution fragment H of M such that Cones(H) is a characterization of ei(H) as a union of 
disjoint cones, i.e., ei(H) = U ge cw es (mC g , and for each qi,q 2 G Cones(H), if gi 7^ ^7 then 
C gi fl Cg 2 = 0. Informally, Cones(H) identifies the points where the event denoted by ei(H) is 
satisfied, also called points of satisfaction. 

The concatenation e\ oc ones e 2 of e\ and e 2 via Cones is the function e such that, for each 
probabilistic execution fragment if of M, 

e(H) = \J e 2 (H\q). (5.13) 

gG Cones(H) 

Proposition 5.3.1 The concatenation of two event schemas is an event schema. That is, if 
e = e\ o Cones e 2> then e is an event schema. 

Proof. Consider a probabilistic execution fragment H. From Proposition 4.2.11 each set 
e 2{H\q) is an event of Tu- From the closure of a u-field under countable union, e(H) is an 
event of Tu- ■ 

Proposition 5.3.2 P H \e x o Cones e 2 (H)] = J2 q eCone.s(H) p H[C q ]P H \ q [e2(H\q)]. 

Proof. Since Cones(H) represents a collection of disjoint cones, from (5.13) we obtain 

P H [ei ocones e 2 (H)]= J2 PH[e 2 (S\q)]. (5.14) 

gG Cones(H) 

From Proposition 4.2.11, for each q £ Cones(H) 

P H [e 2 {H\q)\ = P H [C q ]P Hlq [e 2 (H\q)}. (5.15) 

By substituting (5.15) in (5.14) we obtain the desired result. ■ 

5.3.2 Execution-Based Event Schemas 

Our definition of an event schema is very general; on the other hand, most of the existing 
work on randomized algorithms is based on a very simple rule to associate an event with each 
probabilistic execution. Namely, a set of execution fragments of M is chosen beforehand, and 
then, given a probabilistic execution fragment H , the event associated with H is the ~ 0#. 
We call this class of event schemas execution-based . We have chosen to give a more general 
definition of an event schema for two main reasons: 

1. The concatenation Theorem of Section 5.4.1 (Theorem 5.4.2) does not rely on the fact that 
an event schema is execution-based, but rather on the fact that it is finitely satisfiable. 
Thus, if in the future some different kinds of event schemas will become relevant, here we 
have already the machinery to deal with them. 

2. The event schemas that we use later to define a progress statement (cf. Section 5.5) are 
not execution-based according to the informal description given above. Specifically, the 
start state of a probabilistic execution fragment of M is a finite execution fragment of 
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M, i.e., it contains some history of M, and such history is not considered in determining 
whether there is some progress. On the other hand, it is plausible that sometimes we 
want to consider also the history encoded in the start state of a probabilistic execution 
fragment. Thus, the more general definition of an event schema still helps. 

Nevertheless, execution-based adversary schemas are easier to understand and enjoy properties 
that do not hold for general adversary schemas (cf. Section 5.7). For this reason we give 
a formal definition of an execution-based adversary schema, where we also assume that the 
history encoded in the start state of a probabilistic execution fragment is eliminated. 

Let be a set of extended execution fragments of M. An event schema e for a probabilistic 
automaton M is Q-based iff for each probabilistic execution fragment H of M, e(H) = n 
(flff>qQ ). An event schema e for a probabilistic automaton M is execution-based iff there exists 
a set of extended execution fragments of M such that e is ©-based. 

5.4 Probabilistic Statements 

Given a probabilistic automaton M, an event schema e, an adversary A, and a finite execution 
fragment a, it is possible to compute the probability P vreX ec(M,A,a)[ e (P rexec (M,A,ot))] of the 
event denoted by e when M starts from a and interacts with A. As a notational convention, 
we abbreviate the expression above by P]\4,A,a[ e ]- Moreover, when M is clear from the context 
we write P^afe], and we write Pyt[e] if M has a unique start state and a is chosen to be the 
start state of M. 

We now have all the machincery necessary to define a probabilistic statement. A probabilistic 
statement for a probabilistic automaton M is an expression of the form Pr^^ Si ©(e) 1Z p, where 
Advs is an adversary schema of M, is a set of starting conditions, i.e., a set of finite execution 
fragments of M, e is an event schema for M, and 1Z is a relation among =, <, and >. A 
probabilistic statement Pr^^ Si ©(e) 1Z p is valid for M iff for each adversary A of Advs and each 
starting condition a of 0, P^afe] 1Z p, i.e., 

Pr^rf ra ,e(e) H p iff ^AeAdvs^ a ee p A, a [ e ] n P- (5.16) 

Proposition 5.4.1 Some trivial properties of probabilistic statements are the following. 

1. If p\ 1Zp 2 then Fr AdvSt e(e) 1Z p 1 implies Fr AdvSt@ (e) 1Z p 2 . 

2. If Advs i C Advs 2 and ©i C 2 , then Pr^^ Sli © 1 (e) 1Z p implies Pr^ ( f VS2j e 2 ( e ) 1Z p. ■ 

5.4.1 The Concatenation Theorem 

We now study an important property of probabilistic statements applied to the concatenation 
of event schemas. Informally, we would like to derive properties of the concatenation of two 
event schemas from properties of the event schemas themselves. The idea that we want to 
capture is expressed by the sentence below and is formalized in Theorem 5.4.2. 

LL Ife\ is satisfied with probability at least pi, and from every point of satisfaction of 
e\, ei is satisfied with probability at least p 2 , then the concatenation of e\ and e 2 is 
satisfied with probability at least pip 2 - v 
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Theorem 5.4.2 Consider a probabilistic automaton M . Let 
1- Pr^ ra ,e(ei) H P\ and , 

2. for each A G AdvS, q£®, let ?T A dvs,Cones( P rexec(M,A,q))( e 2) K P2- 

Then, Pr^^ Si ©(ei o Cones e 2 ) Up 1 p 2 . 

Proof. Consider an adversary A G Advs and any finite execution fragment q G 0. Let 
H = prexec(M,A,q). From Proposition 5.3.2, 

P H [ei ocones e 2 (H)}= J2 PH[C q >]P HW [e 2 (H\q')]. (5.17) 

q'eCones(H) 

Consider an element q' of Cones(H). It is a simple inductive argument to show that 

H\q' = prexec(M,A,q'). (5.18) 

Thus, from our second hypothesis, 

P H \A e 2(H\q')]np 2 . (5.19) 

By substituting (5.19) in (5.17), we obtain 

P H [ei ocones e 2 (H)]TZ P2 J2 p H[C q >]- (5.20) 

q'eCones(e 1 (H)) 

By using the fact that Cones(H) is a characterization of ei(H) as a disjoint union of cones, 
Equation (5.20) can be rewritten into 

P H [ei ocones e 2 (H)]np2P H [ei(H)]. (5.21) 
From the first hypothesis, Pff[ei(if)] 1Z pi; therefore, from Proposition 5.4.1, 

Pn[ei ocones e 2 (H)]TZp 1 p 2 . (5.22) 

This completes the proof. ■ 

5.5 Progress Statements 

In this section we give examples of probabilistic statements, which we call progress statements, 
that play an important role in the analysis of algorithms. Progress statements are formaliza- 
tions of statements that are used generally for the informal analysis of randomized algorithms; 
however, many other statements can be defined depending on specific applications. We show 
also how to derive complex statements by concatenating several simple statements. 
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5.5.1 Progress Statements with States 

Let U and U' be sets of states of a probabilistic automaton M. A common informal statement 
is the following. 

" Whenever the system is in a state of U , then, under any adversary A of Advs, the 
probability that a state of U 1 is reached is at least p." 

The probability p is usually 1. In this thesis we consider the more general statement where p 
is required only to be greater than 0. We represent the statement concisely by writing 

U -^Advs U', (5.23) 

where Advs is an adversary schema. We call (5.23) a progress statement since, if we view U' as 
a better condition than U, then (5.23) states that from U it is possible to have some progress 
with probability at least p. The reader familiar with UNITY [CM88] may note that a progress 
statement is a probabilistic generalization of the leads-to operator of UNITY. 

Let us concentrate on the formal meaning of (5.23). Let eyi be an event schema that given 
a probabilistic execution fragment H returns the set of extended executions a of Q,jj such that 
a state of U' is reached in a>q^ (recall that q$ is the start state of H). Then (5.23) is the 
probabilistic statement 

P^Adv S ,u(eu') > P- (5.24) 

Note that the starting conditions of statement (5.24) are just states of M, i.e., they do not 
contain any past history of M except for the current state. This is because when we reason 
informally about algorithms we do not talk usually about the past history of a system. However, 
if we want to concatenate two progress statements according to Theorem 5.4.2, then we need to 
consider the past history explicitly, and thus a better probabilistic statement for (5.23) would 
be 

FTAdvs&uieu 1 ) > P, (5.25) 

where Qjj is the set of finite execution fragments of M whose last state is a state of U . So, why 
can we, and indeed do people, avoid to deal with the past history explicitly? The point is that 
(5.24) and (5.25) are equivalent for most of the adversary schemas that are normally used. 

5.5.2 Finite History Insensitivity 

An adversary schema Advs for a probabilistic automaton M is finite-history-insensitive iff 
for each adversary A of Advs and each finite execution fragment a of M, there exists an 
adversary A 1 of Advs such that for each execution fragment a' of M with fstate(a') = Istate(a), 
A' (a') = A(a^ a'). In other words, A' does even though A' does not know the finite history a. 

Lemma 5.5.1 Let Advs be a finite-history-insensitive adversary schema for a probabilistic au- 
tomaton M. Then (5.24) an( ^ (5.25) are equivalent probabilistic statements. 
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Proof. From Proposition 5.4.1, since U C 0jy, Statement (5.25) implies Statement (5.24) 
trivially. Conversely, suppose that Statement (5.24) is valid. Consider an adversary A of Advs, 
and consider an element q of Qjj. Let A q be an adversary of Advs such that for each execution 
fragment q' of M with fstate(q') = Istate(q), A q {q r ) = A(q~ q'). We know that A q exists since 
Advs is finite-history-insensitive. It is a simple inductive argument to show that 

prexec(M, A q , Istate(q)) = prexec(M, A, q)>q. (5.26) 

Moreover, 

Pprexec(M,A,q)[Cq] = 1- (5.27) 

From the definition of ejji, since the start state of prexec(M, A, q) is q, 

ejjt(prexec(M,A q A s i a i e {q))) = ejjt(prexec(M,A,q))>q. (5.28) 

Thus, from Proposition 4.2.12 and (5.27), 

PA, q [eu>] = PA q ,lsute(g)[ e U']- (5-29) 

From hypothesis, 

PA q ,lsute(g)[ e U'] >P, (5-30) 

and thus, from (5.29), PA,q[ e U'] > P- This shows that Statement (5.25) is valid. ■ 

5.5.3 The Concatenation Theorem 

We now start to compose (simple) progress statements to derive other (more complex) progress 
statements. This allows us to decompose a complex problems into simpler problems that can be 
solved separately. The examples of Chapter 6 contain explicit use of the concatenation theorem 
of this section. 

Suppose that from U we can reach U' with probability at least p, and that from U' we 
can reach U" with probability at least p' . Then, it is reasonable that from U we can reach U" 
with probability at least pp'. This result is an instantiation of the concatenation theorem of 
Section 5.4.1. 

Theorem 5.5.2 Let Advs be a finite-history-insensitive adversary schema. Then, 
U — >Adv.s U' and U' —>Advs U" imply U — fAdvs U" . 

V p pp 

Proof. Consider the event schemas eyi and ejy». Let Cones be the function that associates 
with each probabilistic execution fragment H the set 

Cones(H) = {q \ lstate(q>q ) £ U' , / 3 g / < ( gI>go ) Istate(q') £ U'}. (5.31) 

It is easy to check that Cones(H) is a characterization of eyi as a disjoint union of cones. Then, 
directly from the definitions, for each execution fragment H , 

ew ° Cones eu"(H)Ceu»(H). (5.32) 
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Informally, the left expression represents the property of reaching a state of U" passing through 
a state of U' , while the right expression represents the property of reaching a state of U" without 
passing necessarily through a state of U' . 

From Lemma 5.5.1, for each probabilistic execution fragment H , each adversary A of Advs, 
and each element q of Cones(H), since Istate(q) £ U', 

PA, q [ e u»]>p'- (5.33) 

From hypothesis, (5.33), and Theorem 5.4.2 (concatenation of two event schemas), 

F*Advs,u(eU' ° Cones ^U") > PP' ■ (5.34) 

From (5.32) and (5.34), 

F*Advs,u(eu") > PP '■ (5.35) 

This shows that U — Advs U" . ■ 

pp' 

Proposition 5.5.3 Other trivial properties of progress statements are the following. 

1. U — > U. 

l 

2. If Ui — > U{ and U 2 — > V' 2 , then TJ X U U 2 — > U{ U V' 2 . ■ 

P\ P2 min(pi,p2) 

5.5.4 Progress Statements with Actions 

Progress statements can be formulated also in terms of actions rather than states. Thus, if V 
is a set of actions, we could write 

U -^Advs V (5.36) 

meaning that starting from any state of U and under any adversary of Advs, with probability at 
least p an action from V occurs. Formally, let ey be an event schema that given a probabilistic 
execution fragment H returns the set of executions a of Q,jj such that an action from V occurs 
in a>qQ . Then (5.36) is the probabilistic statement 

P^Advs,u(ev) > P- (5.37) 

Similarly, we can change the left side of a progress statement. Thus, we can write 

V -^Advs U (5.38) 

p • ' 

meaning that starting from any point where an action from V occurred and no state of U is 
reached after the last occurrence of an action from V , a state of U is reached with probability 
at least p. In other words, after an action from V occurs, no matter what the system has 
done, a state of U is reached with probability at least p. Formally, let ®y,u be the set of finite 
execution fragments of M where an action from V occurs and no state of U occurs after the 
last occurrence of an action from V. Then (5.38) is the probabilistic statement 

FTAdvs,e VtU (eu) > P- (5.39) 
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Finally, we can consider statements involving only sets of actions. Thus, the meaning of 
V — >Advs V would be the probabilistic statement 

FTAdvs,e ViV ,(ev) > P, (5.40) 

where ®yy 1S the set of finite execution fragments of M where an action from V occurs and 
no action from V occurs after the last occurrence of an action from V. 

The concatenation theorem extendeds easily to the new kinds of progress statements. 

Theorem 5.5.4 Let Advs be a finite-history-insensitive adversary schema, and let X,X' and 
X" be three sets, each one consisting either of actions of M only or states of M only. Then, 

X — >Adv.s X' and X' — >Adv.s X" imply X — >Adv.s X" . 

Pi '92 Pl'P2 

Proof. This proof is similar to the proof of Theorem 5.5.2, and thus it is left to the reader. 
Observe that finite-history-insensitivity is not necessary if X' is a set of actions. ■ 

5.5.5 Progress Statements with Probability 1 

Usually we are interested in progress properties that hold with probability 1. A useful result is 
that in most cases progress with probability 1 can be derived from progress with any probability 
p such that < p < 1. Specifically, under the condition that an adversary never chooses 8 when 
the left side of a given progress statement is satisfied and the right side of the same progress 
statement is not satisfied, 

1. if the left element of the progress statement is a set of actions, then progress is achieved 
with probability 1; 

2. if the left element of the progress statement is a set of states U, the adversary schema is 
finite-history-insensitive, and the system remains in a state of U unless the right side of 
the statement is satisfied, then progress is achieved with probability 1. 

Proposition 5.5.5 Suppose that V — >Advs X , and suppose that 8 ^ &A(q) f or ea °h adversary 
A of Advs and each element q of ®y,x- Then V — >Advs X . 

Proof. We give the proof for the case where X is a set of states. The other proof is similar. 
Denote X by U . 

Consider an element go of ®v,u an d an adversary A of Advs. Let H be prexec(M,A,qo), 
and let p' = PH[ejj{H)]. We know from hypothesis that p' > p. Suppose by contradiction that 
p' < 1. Let be the set of finite execution fragments q of M such that go < Q, Istate(q) £ U, 
and no state of U occurs in any proper prefix of q>qo. Then is a characterization of ejj(H) 
as a union of disjoint cones. Thus, 

P H [eu(H)]=Y J PH[C q ]. (5.41) 

gee 

Let e be any real number such that < e < p'. Then, from (5.41) and the definition of p', it is 
possible to find a natural number k e such that. 



E 

gee|| g |<fc £ 



P H [C q ]>(p'-e). (5.42) 
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Let © e be the set of states q of H such that \q\ = k e and no prefix of q is in 0. That is, © e is 
the set of states of H of length k e that are not within any cone C q of ejj(H) where \q\ < k e . 
Equation (5.41) can be rewritten as 

P H [eu(H)] = I J2 P H[C q ]) + ( E P H [C q ]P H [eu(H)\C q ]\ . (5.43) 

\gee|| g |<fc £ / \gee e / 

Observe that for each state q of © e , since a state of U' is not reached yet, q is an element of ®y,u- 
Moreover, prexec(M,A,q) = H\q (simple inductive argument). Thus, from Proposition 4.2.11 
and hypothesis, PH[ e u{H)\C q ] > p, and (5.43) can be rewritten into 

P H [eu(H)] > ( J2 P H[C q ]) + f E P H[C q ]p] ■ (5.44) 

\gee|| g |<fc £ / \qee e / 

Observe that J2qe&\\q\<k e Pn[C q ] + J2 q e&e P H[C q ] = 1. This follows from the fact that if a state 
q of H does not have any prefix in 0, then q £ ®v,x, which in turn means that 6 £ 0^. In 
other words, in H it is not possible to stop before reaching either a state of {q £ | \q\ < k f } 
or a state of e . Thus, by using (5.42) in (5.44) we obtain 

P H [eu(H)} > (p 1 - e) + (1 - (p 1 - e))p. (5.45) 

After simple algebraic manipulations, Equation (5.45) can be rewritten into 

P H [eu(H)} >p' + p(l - p 1 ) - e(l - p). (5.46) 

If we choose e such that < e < p(l—p')/(l—p), which exists since p' < 1, then Equation (5.46) 
shows that Pff[e[/(if)] > p' . This contradicts the fact that p' < 1. Thus, Pff[e[/(if)] = 1. ■ 

For the next proposition we define the statement U Unless X , where U is a set of states and X 
is either a set of states only or a set of actions only. The statement is true for a probabilistic 
automaton M iff for each transition (s, 7-*) of M, if s £ U — X then for each (a, s') £ fi either 
a£X,ors'£ U U X . That is, once in U, the probabilistic automaton M remains in U until 
the condition expressed by X is satisfied. 

Proposition 5.5.6 Suppose that U — >Advs X, U Unless X , Advs is finite-history-insensitive, 

and S £ fiyt(s) for each adversary A of Advs and each state s of U . Then, U — >Advs X . 

Proof. This proof is similar to the proof of Proposition 5.5.5. The main difference is that the 
passage from Equation (5.43) to Equation (5.44) is justified by using finite-history-insensitivity 
as in the proof of Proposition 5.5.1. ■ 

5.6 Adversaries with Restricted Power 

In Section 5.2 we have defined adversary schemas to reduce the power of an adversary; however, 
we have not described any method to specify how the power of an adversary is reduced. In 
this section we show two methods to reduce the power of an adversary. The first method, 
which is the most commonly used, reduces the kind of choices that an adversary can make; 
the second method, which is used in informal arguments but is rarely formalized, reduces the 
on-fine information used by an adversary to make a choice. The two specification methods are 
used in Section 5.7 to study the relationship between deterministic and randomized adversaries. 
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5.6.1 Execution-Based Adversary Schemas 

If n processes run in parallel, then a common requirement of a scheduler is to be fair to all the 
processes. This means that whenever an adversary resolves the nondeterminism and leads to 
a probabilistic execution fragment H , in all the executions of Q,jj each one of the n processes 
performs infinitely many transitions. More generally, a set of extended execution fragments 
of M is set beforehand, and then an adversary is required to lead only to probabilistic execution 
fragments whose corresponding sample space is a subset of 0. 

Formally, let be a set of extended execution fragments of M. Let Advs@ be the set of 
adversaries A such that for each finite execution fragment q of M, $l vreX ec(M,A,q) — ©• Then 
AdvsQ is called 0-based. An adversary schema Advs is execution-based iff there exists a set 
of extended execution fragments of M such that Advs is 0-based. 

The notion of finite-history-insensitivity can be reformulated easily for execution-based ad- 
versary schemas. Define to be finite-history-insensitive iff for each extended execution frag- 
ment a of M and each finite execution fragment a' of M such that Istate(a') = fstate(a), if 
a' ~ a £ then a £ 0. It is easy to verify that if is finite-history-insensitive, then AdvsQ is 
finite-history-insensitive. 

5.6.2 Adversaries with Partial On-Line Information 

Sometimes, like in the case of the toy resource allocation protocol, an adversary cannot base 
its choices on the whole history of a system if we want to guarantee progress. In other words, 
some part of the history is not visible to the adversary. 

Example 5.6.1 (Off-line scheduler) The simplest kind of adversary for n processes that run 
in parallel is an adversary that fixes in advance the order in which the processes are scheduled. 
This is usually called an off-line scheduler or an oblivious adversary . Thus, at each point a 
the next transition to be scheduled depends only on the ordered sequence of processes that are 
scheduled in a. 

To be more precise, the transition scheduled by the adversary depends also on the state that 
is reached by a, i.e., Istate(a), since a specific process may enable different transitions from 
different states. This means that if a\ and a.^ are equivalent in terms of the ordered sequence 
of processes that are scheduled, the oblivious constraint says only that the transitions chosen 
by the adversary in a.\ and a.^ must be correlated, i.e., they must be transitions of the same 
process. ■ 

The formal definition of an adversary with partial on-line information for a probabilistic au- 
tomaton M is given by specifying two objects: 

1. an equivalence relation that specifies for what finite execution fragments of M the choices 
of an adversary must be correlated; 

2. a collection of correlation functions that specify how the transitions chosen by an adver- 
sary must be correlated. 

Let = be an equivalence relation between finite execution fragments of M, and let F be a 
family of functions parameterized over pairs of equivalent execution fragments. Each function 
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f aa i takes a combined transition of M leaving from Istate(a) and returns a combined transition 
of M leaving from Istate(a') such that 

1- fa'a(faa'(tr)) = tr; 

2. f aa >(T,ieiPi tr i) = T,ieiPifoa'(tri). 

The pair ( = , F) is called an oblivious relation. An adversary A is oblivious relative to ( = , F) iff 
for each pair of equivalent execution fragments of M, a = a' , A(a') = faa'(A(a)). An adversary 
schema Advs is said to be with partial on-line information iff there exists an oblivious relation 
( = ,F) such that Advs is the set of adversaries for M that are oblivious relative to ( = ,_F). 

Condition 1 is used to guarantee that there are oblivious adversaries relative to ( = ,_F); 
Condition 2 is more technical and is used to guarantee that there are oblivious adversaries 
relative to ( = ,_F) that do not use randomization in their choices. Condition 2 is needed mainly 
to prove some of the results of Section 5.7. 

Adversaries with partial on-line information and execution-based adversaries can be com- 
bined together easily. Thus, an adversary schema Advs is said to be execution-based and with 
partial on-line information iff there exists an execution-based adversary schema Advs' and a 
pair ( = , F) such that Advs is the set of adversaries of Advs' that are oblivious relative to ( = , F). 

Example 5.6.2 (Adversaries for the toy-resource allocation protocol) The fair obliv- 
ious adversaries for the toy resource allocation protocol are an example of an execution-based 
adversary schema with partial on-line information. The set is the set of executions of Mi||M2 
where both Mi and Mi perform infinitely many transitions. Two finite execution fragments 
a.\ and a.2 are equivalent iff the ordered sequences of the processes that perform a transition 
in a.\ and a.^ are the same. Let a.\ = a.^, and let, for i = 1,2, fr^i and tr-{^ be the tran- 
sitions of M\ and M2, respectively, enabled from Istate(ai). Then f ai a 2 {t r i,i) = ^ r 2,i and 

f aia2 ( tr i,2) = tr 2,2- 

Another execution-based adversary schema with partial on-line information that works for 

the toy resource allocation protocol is obtained by weakening the equivalence relation so that 

an adversary cannot see only those coins that have not been used yet, i.e., those coins that have 

been flipped but have not been used yet to check whether the chosen resource is free. ■ 

5.7 Deterministic versus Randomized Adversaries 

In our definition of an adversary we have allowed the use of randomness for the resolution of 
the nondeterminism in a probabilistic automaton M. This power that we give to an adversary 
corresponds to the possibility of combining transitions of M in the definition of a probabilistic 
execution fragment. From the formal point of view, randomized adversaries allow us to model a 
randomized environment and to state and prove the closure of probabilistic execution fragments 
under projection (Proposition 4.3.4). However, one question is still open: 

Are randomized adversaries more powerful than deterministic adversaries? 

That is, if an algorithm performs well under any deterministic adversary, does it perform well 
under any adversary as well, or are there any randomized adversaries that can degrade the 
performance of the algorithm? In this section we want to show that in practice randomization 
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does not add any power to an adversary. We say "in practice" because it is easy to build 
examples where randomized adversaries are more powerful than deterministic adversaries, but 
those examples do not seem to be relevant in practice. 

Example 5.7.1 (Randomization adds power) Consider an event schema e that applied to 
a probabilistic execution fragment H returns Q,jj if H can be generated by a deterministic 
adversary, and returns otherwise. Clearly, if M is a nontrivial probabilistic automaton, the 
probability of e is at least 1 under any deterministic adversary, while the probability of e can 
be under some randomized adversary; thus, randomization adds power to the adversaries. 
However, it is unlikely that a realistic event schema has the structure of e. Another less 
pathological example appears in Section 5.7.2 (cf. Example 5.7.2). ■ 

We consider the class of execution-based event schemas, and we restrict our attention to the 
subclass of finitely satisfiable, execution-based event schemas. We show that randomization does 
not add any power for finitely satisfiable, execution-based event schemas under two scenarios: 
execution-based adversary schemas, and execution-based adversary schemas with partial on-line 
information. In the second case we need to be careful (cf. Example 5.7.2). 

Informally, a randomized adversary can be seen as a convex combination of deterministic 
adversaries, and thus a randomized adversary satisfies the same probability bounds of a deter- 
ministic adversary. However, there are uncountably many deterministic adversaries, and thus 
from the formal point of view some more careful analysis is necessary. 

5.7.1 Execution-Based Adversary Schemas 

Proposition 5.7.1 Let Advs be an execution-based adversary schema for M , and let Advsp 
be the set of deterministic adversaries of Advs. Let e be a finitely- satisfiable, execution-based, 
event schema for M . Then, for every set of finite execution fragments of M , every probability 
p, and every relation 1Z among <, =, >, FrAdvs,e( e ) T^P iff ^ T Advs D ,&( e ) 1Z p. ■ 

In the rest of this section we prove Proposition 5.7.1. Informally, we show that each probabilistic 
execution fragment H generated by an adversary of Advs can be converted into two other 
probabilistic execution fragments H' and H", each one generated by some adversary of Advsp, 
such that P H ,[e(H')] < P H [e(H)] < P H ,,[e(H")]. Then, if K is < we use H" , and if K is > we 
use H' . 

An operation that is used heavily in the proof is called deterministic reduction. Let if be a 
probabilistic execution fragment of a probabilistic automaton M, and let q be a state of H . A 
probabilistic execution fragment H' is said to be obtained from H by deterministic reduction 
of the transition enabled from q if H' is obtained from H through the following two operations: 

1. Let tr^ = q ~ (J2ieiPit r i) where each pi is non-zero and each tr{ is a transition of M. 
Then replace tr^ either with (q,V(Sj) or with q" trj, under the restriction that (q,V(Sj) 
can be chosen only if J2ieiPi < 1- 

2. Remove all the states of H that become unreachable after tr^ is replaced. 

Throughout the rest of this section we assume implicitly that whenever a probabilistic execution 
fragment is transformed, all the states that become unreachable are removed. 
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Lemma 5.7.2 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based event schema such that Pjj\e(^H)\ = p. Let q be a state 
of H . Then there exist two probabilistic execution fragments Hj ow ,Hl ih , each one generated 
by an adversary of Advs, that are obtained from H by deterministic reduction of the transition 
enabled from q, and such that P H q [e(Hf ' )] < p, and P H q [ e (Hl ih )] > p. 

low high y 

Proof. Let tr^ be g" (J2ieiPit r i)i where each tr{ is either a transition of M or the pair 
(lstate(q),V(6)), each pt is greater than 0, and J2ielPi = 1- -^ or eacn transition tr\, i £ I, let 
Hf ri be obtained from H by replacing tr^ with q^ tr{. Observe that, since Advs is execution- 
based and H is generated by an adversary of Advs, H iri is generated by an adversary of Advs. 
The probability of e(H) can be written as 

P H [e(H)} = P H [C q ]P H [e(H)\C q ] + (1 - P H [C q ])P H [e(H)\C^. (5.47) 

Observe that for each i £ /, since H and H iri differ only in the states having q as a prefix, 
PH[C q ] = PH tr .[C q ]. Since e is execution-based, e(H)P\C q = e(H tri )C\C q , and Pu\e{H) PiC q ] = 
PH tr \ e {Htri) H C q ] (use conditional probability spaces and Theorem 3.1.2). Moreover, as it is 
shown below, P H [e(H) l~l C q ] = Y<ieiPiPH tr . [e(H tri ) n C q ]. In fact, 

P H [e(H)nC q ] = P H [C q ] Pq H [S]P H [e(H)\Cq S ] + J2 Pq[(^q')]PH[e(H)\Cq,} ,(5.48) 

V (a,9')enf / 

where we assume that PH[e{H)\C q $\ is whenever it is undefined. For each (a,q') of £1? , 

P q [(q,a')] = J2ielPiP<i r '[( a i < l')]i an( i f° r eac h i such that (a,q r ) £ fl q r ' , Pjj[e(H)\C q i] = 
PH tr [ e (Htr,)\C q i] (simply observe that H>q' = H tr ,>q')- Similarly, if S £ 0^, then P^[S] = 

J2 t eiPi P i tr '[^]' and for eacn * sucn tnat ^ G fig "\ Pff[e(if)|C g ,5] = PH tr .[e(H tri )\C q 8\. Thus, 
from (5.48), 

P ff [e(i7)nC g ] = ^p 8 P fftr JC g ] 
iei 
( \ 



(5.49) 



^ tr ^]PH tri [e(H trt )\C qS ] + J2 P * tr 'l(^l')}PH tr M H ^)\C q i 
\ (a, q ')en" tr ' 

which gives the desired equality 

P H [e(H) n C q ] = J2Pi p H trt HHtr t ) n C q ]. (5.50) 

iei 

Thus, (5.47) can be rewritten into 



PhHH)] = $> [P Htri [Cq]P Htri [e(H tri )\Cq] + (1 - PH tri [C q ])PH tri [e(H tri )\C q ]) , (5.51) 
iei 



which becomes 



PhHH)] = S>iVje(# iri )]. (5.52) 

iei 
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If there exists an element i of L such that Pu tr [ e (Htr t )] = P, then fix Hf ow and H^, to be H iri . 
If there is no element i of L such that Pu tr [ e (Htr t )] = P, then it is enough to show that there 
are two elements i\,ii of/ such that Pu tr [ e (Htr t )] < P an d Pn tr . [ e (Htr t )] > P, respectively. 
Assume by contradiction that for each element i of /, Pu tr [ e (Htr t )] < P- Then, from (5.52), 
J2ieiPiPH tr \ e (Htri)] < P, which contradicts Pff[e(if)] = p. Similarly, assume by contradiction 
that for each element i of I, PH tri [e(H tri )] > V- Then, from (5.52), 2; e /P;-Pff tr . [e(Htr,)] > P, 
which contradicts Pfj[e(H)] = p again. ■ 

Lemma 5.7.3 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based event schema such that Pfj[e(II)] = p. Let d be a natural 
number, and let JJ,i be the set of states q of H such that \q\ = d. Then there exist two probabilistic 
execution fragments Hi 0W ,Hki g h, each one generated by an adversary of Advs, that are obtained 
from H by deterministic reduction of the transitions enabled from the states of JJ,i, and such 
that P Hlow [e(Hi 0W )] < p, and PH high [e(H high )] > p. 

Proof. From Lemma 5.7.2 we know that for each state q of JJ,i there are two probabilistic exe- 
cution fragments Hj ow and H^,, obtained from H by deterministic reduction of the transition 
enabled from q, such that P H q [e(Hf ' )] < p, and P H q [ e (-H7, a /,)] ^ P- Let H\ ow be obtained 

low high y 

from H by replacing the transition enabled from each state q of JJ,i with the transition enabled 
from q in Hj ow , and let Hhigh be obtained from H by replacing the transition enabled from each 
state q of JJ,i with the transition enabled from q in H^,. Since Advs is execution-based and 
all the involved probabilistic execution fragments are generated by an adversary of Advs, then 
Hhigh an( i H\ ow are generated by an adversary of Advs. Since e is execution-based, for each 
state q of U d , P Hlow [e(H low ) n C q ] = P H , g Je(HlJ n C q }. Thus, 

P Hlow [e(H l0W )} = £ PH l0W [C q ]P Hl Je(Hl w )\C q }. (5.53) 

qeu d 

Observe that, for each state q of Ud, the difference between the probability of e(H) and the 
probability of e(Hj ow ) is determined by the subcones of C q . Thus, 

P Hlow [e(H l0W )] < J2 P H [C q ]P H [e(H)\C q ]. (5.54) 

q£U d 

The right side of (5.54) is Pfj[e(II)], which is p. In a similar way it is possible to show that 
p H high [<H high )]>p. M 

Now we use the fact that e is finitely satisfiable. For each probabilistic execution fragment H 
of M , let Can(e(H)) the set of minimal elements of {q £ states(H) \ C q C e(H)} U {qS \ q £ 
states(H),C q s ^ e(H)}. Then, Can(e(H)) is a characterization of e(H) as a union of disjoint 
cones. For each natural number d, let e\d be the function that given a probabilistic execution 
fragment H returns the set ^ qe can(e(H))\\q\<dC q ■ 

Lemma 5.7.4 Let e be an execution-based, finitely satisfiable, event schema for a probabilistic 
automaton M , and let d, d! be two natural numbers such that d < d! . Then, for each probabilistic 
execution fragment H, P H [e\d(H)] < P H [e\d'(H)] < P H [e(H)]. 
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Proof. Follows trivially from the definitions. ■ 

Lemma 5.7.5 Let e be an execution-based, finitely satisfiable, event schema for a probabilistic 
automaton M , and let d be a natural number. Let H be a probabilistic execution fragment H 
of M , and let H' be obtained from H by reducing deterministically any collection of states of 
length greater than d. Then, Pjj\e\d(^H)\ < Pjji[e\d(H')]. 

Proof. Just observe that for each q £ Can(e(H)) such that \q\ < d there is a q' £ Can(e(H')) 
such that q' < q, and that for each state q of H such that |g| < d, PH[C q ] = PH'[Cq]- ■ 

Lemma 5.7.6 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based, finitely satisfiable event schema such that Pfj[e(II)] = p. 
Then there exists a probabilistic execution fragment H' , generated by a deterministic adversary 
of Advs, such that Pjji[e(H')] < p. 

Proof. From Lemma 5.7.3 it is possible to find a sequence of probabilistic execution fragments 
(Hi)i>o, where Ho = H , each -ff;+i is obtained from Hi by deterministically reducing all its 
transitions leaving from states of length i, and for each i, Ph z+1 [ e (Hi+i)] < -Pff;[ e (-^«')]- Let H' 
be obtained from H by replacing the transition enabled from each state q with the transition 
enabled from q in any Hi such that \q\ < i. It is immediate to check that H' is generated by 
some deterministic adversary of Advs (every extended execution of 0#/ is an extended execution 
of fiff). 

Suppose by contradiction that Pfji[e(H' )] > p. Then there exists a level d such that 

P H ,[e\d(H')]>p. (5.55) 

For each d! > d, let E^ be 

E d , = U C f- ( 5 - 56 ) 

qeCan(etd'(H d ,))\3 q , eCan(etd(Hl)) q'<q 

Then, the following properties are valid. 

1. for each d! > d, E' d is an element of J~h'- 
Ed' is a union of cones of J~h'- 

2. if d' < d", then E d , C E d „ 

Consider an element q £ Can(e\d'(H d ')) such that there exists a q' £ Can(e\d(H')) such 
that q' < q. Observe that, since H^' is obtained from H^ by deterministic reduction of 
states of length greater than d' , there exists a q" £ Can(e\d"(H d ")) such that q" < q. 
Moreover, from the construction of H\ q' < q". Thus, from (5.56), Czf, C Ed". Since 

l" ^ c h C? C Ed", and therefore, Ed' C Ed". 

3. e\d(H')C\J d >> d E d >. 

Consider an element a of e\d(H r ). Then, for each d' , a £ e(Hd'). Let q' £ Can(e(Hd)) 
such that q' < a, and let d! be |g'|. Then, there exists a q" £ Can(e\d l (Hd')) such that 
q" < q' < a, and thus a £ E^. 
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4. for each d! > d, P Hdl [e\d'(H d ,)} > P H >[E d >]. 

From the construction of if', for each q such that \q\ < d', Pjj ,[C q d '] = Pjji[Cg ]. 
Moreover, if Cg is used in the definition of E^, then q G Can(e\d'(H d i)). 

From 2 and 3, and from (5.55), there exists a value d! such that Ph'[E<1'] > P- From 4, 
Ph [e\d'(Hd')] > P- From Lemma 5.7.4, PH d ,[ e (Hd')] > P- This contradicts the fact that 
P Hd ,[e\d'(H d ,)} < p. M 

To build a probabilistic execution fragment H', generated by an adversary of Advsp, such that 
Pfji[e(H')] > p, we need to extend part of Lemmas 5.7.2 and 5.7.3. 

Lemma 5.7.7 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary of 
Advs. Let e be an execution-based, finitely- satisfiable, event schema. Let q be a state of H , and 
let d be a natural number such that Pu\e\d(H)\ = p. Then there exist a probabilistic execution 
fragment H^,, generated by an adversary of Advs, that is obtained from H by deterministic 
reduction of the transition enabled from q, such that P H q [e \d{Hl ih )] > p. 

high y 

Proof. This proof is similar to the proof of Lemma 5.7.2, with the difference that the = sign 
of Equations (5.49), (5.50), (5.51), and (5.52), is changed into a <. In fact, in each one of the 
Htr t some new cone of length at most d may appear. ■ 

Lemma 5.7.8 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based, finitely- satisfiable, event schema, and let d be a natural 
number such that Ph[c \d(H)] = p. Let d! be a natural number, and let Ud> be the set of states q 
of H such that \q\ = d! ' . Then there exist a probabilistic execution fragment H h i gh , generated by 
an adversary of Advs, that differs from H only in that the transitions enabled from the states 
of Ud are deterministically reduced, such that Ph m h [ e \d(Hhigh)] ^ P- 

Proof. This proof is similar to the proof of Lemma 5.7.3. In this case the arguments for the 
equation corresponding to Equation (5.54) is justified from the additional fact that Hhigh may 
have more cone of depth at most d than H . ■ 

Lemma 5.7.9 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based, finitely- satisfiable, event schema such that Pjj\e(^H)\ > p. 
Then, there exists a probabilistic execution fragment H' of M , generated by a deterministic 
adversary of Advs, such that Pfj[e(II')] > p. 

Proof. Since Pfj[e(II)] > p and e(H) is a union of cones, there exists a natural number d such 
that PH[e\d(H)] > p. From repeated applications of Lemma 5.7.8, one for each level d' < d, 
there exists a probabilistic execution fragment H", obtained from H by deterministic reduction 
of the transitions enabled from every state q with \q\ < d, such that Pjjii[e\d(H")] > p. From 
Lemma 5.7.4, Pjju[e(H")] > p. Moreover, any probabilistic execution fragment H'" obtained 
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from H" by reducing deterministically transitions at depth greater than d (\q\ > d) satisfies 
Pjjttt[e\d(H'")] > p, and thus Pjjiii[e(H"')] > p. Hence, H' can be any probabilistic execution 
fragment obtained from H" by reducing deterministically all the transitions at depth greater 
than d in any arbitrary way. It is easy to check that H' is generated by a deterministic adversary 
of Advs. ■ 

Lemma 5.7.10 Let Advs be an execution-based adversary schema for a probabilistic automaton 
M , and let H be a probabilistic execution fragment of M that is generated by some adversary 
of Advs. Let e be an execution-based, finitely- satisfiable, event schema such that Pfj[e(II)] > p. 
Then, there exists a probabilistic execution fragment H' of M , generated by a deterministic 
adversary of Advs, such that Pfj[e(H')] > p. 

Proof. If Pfj[e(H)] > p, then Lemma 5.7.9 suffices. If Pfj[e(H)] = p, then by Lemma 5.7.3 
it is possible to find a sequence of probabilistic execution fragments (Hi)i>o, where Hq = H , 
each -ff;+i is obtained from Hi by deterministically reducing all its i-level transitions, and 
for each i, Ph z+1 i e (Hi+i)] > -fffJ e (-^«)]- If there exists a sequence (Hi)i>o such that for 
some i, Pfj, [e(Hi)] > p, then Lemma 5.7.9 suffices. Otherwise, consider the sequence of 
probabilistic execution fragments defined as follows: Hq = H and, for each i, let d{ be 
the level of Hi such that Pjj i [e\di(Hi)\ > P^j<i(l/2)- 7+1 . Let Hi+i be obtained from re- 
peated applications of Lemma 5.7.8, till level di, so that Ph z+1 [ e \di(Hi + i)] > pJ2j<i(^/2y +1 - 
Note that Ph z+1 [ e (-^8+i)] = Vi otherwise we can find a sequence (Hi)i>o and an i such that 
PH t+1 [ e (Hi+i)] > P (simple argument by contradiction). Let H' be obtained from H by replac- 
ing the transition enabled from each state q with the transition enabled from q in any Hi such 
that \q\ < di-\. It is easy to check that H' is generated by an adversary of Advs. Suppose by 
contradiction that Pfji[e(H' )] = p' < p. Then, from the construction of the H^s, there exists an 
i such that pJ2j<i(^/2y +1 > p' , and thus Pff j+1 [e \di(Hi + i)] > p' . However, from the definition 
of if' , Ph z+1 [ e \di(Hi + i)] = Pfji[e\di(H')], and thus p' < PH>[e(H')], which contradicts the fact 
that P H ,[e(H')]=p'. m 

Proof of Proposition 5.7.1. Since Advsp C Advs, Pr^^ Si ©(e) 1Z p implies Pr^^ SDi 0(e) 1Z p 
trivially. Conversely, suppose that Pr^^ SDi 0(e) 1Z p, and let if be a probabilistic execution 
fragment, generated by an adversary of Advs, whose start state is in 0. We distinguish the 
following cases. 

1. ^is >. 

From Lemma 5.7.6, there is a probabilistic execution fragment H' , generated by an ad- 
versary of Advs£>, whose start state is in 0, and such that Pjji[e(H')] < Ph [e(H )]. From 
hypothesis, P H ,[e(H')] > p. Thus, P H [e(H)] > p. 

2. ^is <. 

From Lemma 5.7.10, there is a probabilistic execution fragment H' , generated by an 
adversary of Advsp, whose start state is in 0, and such that Pff/[e(if')] > Pfj[e(H)]. 
From hypothesis, Pjji[e(H')] < p. Thus, Pfj[e(H)] < p. 

3. ^is =. 

This follows by combining Items I and 2. ■ 
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5.7.2 Execution-Based Adversary Schemas with Partial On-Line Informa- 
tion 

Proposition 5.7.1 can be extended to adversary schemas that do not know all the past history 
of a system, i.e., to execution-based adversary schemas with partial on-line information. We 
need to impose a technical restriction, though, which is that an adversary should always be 
able to distinguish two execution fragments with a different length (cf. Example 5.7.2). The 
proof of the new result is a simple modification of the proof of Proposition 5.7.1. 

Proposition 5.7.11 Let ( = ,_F) be an oblivious relation such that for each pair a\ = a.^ of 
equivalent execution fragment, a\ and a.^ have the same length. Let Advs be an execution- 
based adversary schema with partial on-line information such that each adversary of Advs is 
oblivious relative to ( = ,F), and let Advsp be the set of deterministic adversaries of Advs. 
Let e be a finitely- satisfiable, execution-based, event schema for M . Then, for every set of 
finite execution fragments of M , every probability p, and every relation 1Z among <, =, >, 
Pr AdvS) e(e)TZp iffPr AdvSD) e(e)TZp. 

Proof. The proof is similar to the proof of Proposition 5.7.1. The main difference is in the 
proofs of Lemmas 5.7.2, 5.7.3 and 5.7.8, where equivalence classes of states rather than single 
states only must be considered. In these two proofs we use also the fact that equivalent execution 
fragments have the same length. The details of the proof are left to the reader. ■ 

Example 5.7.2 (Why length sensitivity) The requirement that an adversary should al- 
ways see the length of a probabilistic execution fragment seems to be artificial; however, ran- 
domized adversaries have more power in general if they cannot see the length of a probabilistic 
execution. Consider the probabilistic automaton M of Figure 5-3, and suppose that all the 
executions of M that end in states si, S2, S3, and sq are equivalent. Since for each state s 8 - there 
is exactly one execution of M that ends in s 8 -, we denote such an execution by qj. Let be the 
set of extended executions a 8 of M such that Istate(a) does not enable any transition in M. 
For each state s 8 - that enables some transition, let £r 8jU be the transition that leaves from s 8 - and 
goes upward, and let tr-{ )d be the transition that leaves from s 8 - and goes downward. Then, for 
each pair i,j G {1,2,3,6}, i ^ j, let f qiqj (tr itU ) = tr jtU , and let f qiqj {tr^ d ) = tr J)d . 

Let Advs be the set of 0-based adversaries for M that are oblivious relative to ( = ,_F), and 

let Advs£> be the set of deterministic adversaries of Advs. Then, the statement {so} — >Advs D 

1/2 

{ s 7 5 s w} is valid, whereas the statement {so} — >Advs {$7, sio} is n °t valid, i.e., an adversary can 

1/2 

use randomization to reduce the probability to reach states {s7,sio}- In fact, the probabilistic 

executions Hi and H2 of Figure 5-3 are the only probabilistic executions of M that can be 

generated by the adversaries of Advsp, while Ho is generated by an adversary of Advs. The 

probability of reaching {57,610} in Hi and Hi is 1/2, whereas the probability of reaching 

{•S7,sio} in H is 1/4. ■ 

5.8 Probabilistic Statements without Adversaries 

The current literature on randomized distributed algorithms relies on the notion of an adversary, 
and for this reason all the definitions given in this chapter are based on adversaries. However, 
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Figure 5-3: Randomization adds power for some adversaries with partial on-line information. 

the key objects of the theory that we have presented are the probabilistic execution fragments of 
a probabilistic automaton, and not its adversaries. An adversary schema can be replaced by an 
arbitrary set of probabilistic execution fragments in the definition of a probabilistic statement, 
namely, the set of probabilistic execution fragments that the adversary schema can generate. In 
other words, an adversary schema can be seen as a useful tool to express a set of probabilistic 
execution fragments. 

5.9 Discussion 

Two objects that we have defined in this chapter and that do not appear anywhere in the 
literature are adversary schemas and event schemas. Both the objects are needed because, 
differently from existing work, in this thesis we identify several different rules to limit the 
power of an adversary and several different rules to associate an event with a probabilistic 
execution fragment, and thus we need some way to identify each rule. The best way to think 
of an adversary schema and of an event schema is as a way to denote the rule that is used to 
limit the power of an adversary and denote the rule that is used to associate an event with each 
probabilistic execution fragment. 

We have defined the classes of execution-based adversary schemas and execution-based 
event schemas, and we have proved that for finitely satisfiable execution-based event schemas 
randomization does not increase the power of an execution-based adversary schema, or of a 
class of execution-based adversary schemas with partial on-line information. These results are 
of practical importance because most of the known event schemas and adversary schemas of 
practical interest are execution-based. As a result, it is possible to verify the correctness of 
a randomized distributed algorithm by analyzing only the effect of deterministic adversaries, 
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which is easier than analyzing every adversary. A similar result is shown by Hart, Sharir and 
Pnueli [HSP83] for fair adversaries and almost-sure termination properties, i.e., properties that 
express the fact that under all fair adversaries the system reaches some fixed set of states 
with probability 1. Fair adversaries and termination events are expressible as execution-based 
adversary schemas and finitely satisfiable execution-based event schemas, respectively; thus, 
the result of Hart, Sharir and Pnueli is implied by our result. Hart, Sharir and Pnueli prove 
also that another class of adversaries is equivalent to the class of fair adversaries, namely, those 
adversaries that lead to fair executions with probability 1. The same result holds here as well; 
however, it is not clear under what conditions a similar result holds in general. 
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Chapter 6 

Direct Verication Proving a 
Property 



In this chapter we illustrate techniques to prove the validity of a probabilistic statement from 
scratch. The main technique, which is based on coin lemmas, consists of reducing the analysis of 
a property of a probabilistic automaton to the analysis of a property of an ordinary automaton. 
We illustrate the methodology by applying it to some existing randomized algorithms. 

Part of this chapter is based on joint work with Anna Pogosyants and Isaac Saias. Anna 
Pogosyants suggested us the coin event OCC (Section 6.2.3) as a generalization of other less 
elegant coin events that we had in mind and collaborated on the verification of the randomized 
algorithm for agreement of Ben-Or (Section 6.5). The verification of the randomized dining 
philosophers algorithm of Lehmann and Rabin (Section 6.3) is based on joint work with Nancy 
Lynch and Isaac Saias [LSS94], and the verification of the randomized algorithm for agreement 
of Ben-Or is a formalization of a proof that appears in the book on distributed algorithms of 
Nancy Lynch [Lyn95]. 

6.1 How to Prove the Validity of a Probabilistic Statement 

In Chapter 5 we have defined formally what is a probabilistic statement and we have shown how 
it is possible to combine probabilistic statements to derive more complex properties. However, 
one question is left open: how do we prove the validity of a given probabilistic statement from 
scratch? 

The problem is not trivial: a property may rely on complicate global configurations of a 
system that depend on several separated random draws. Analyzing the exact probability of an 
event associated with a probabilistic execution fragment may be extremely hard. Fortunately, 
there are usually some key points, known to the designer of a system, where specific probabilistic 
choices lead to the desired property. In this chapter we formalize the idea above by introducing 
a collection of coin lemmas. The idea behind a coin lemma is the following. 

1. We define a mechanism to identify events of the kind "some specific probabilistic choices 
yield some specific results". We call such events coin events since a common source of 
randomness is given by coin flips. 
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2. We prove a lower bound on the probability of the coin event that we identify. 

Then, the analysis of a probabilistic statement for a probabilistic automaton M proceeds as 
follows. 

1. We find a coin event that expresses the key intuition behind the property to be shown. 

2. We show that the coin event is a subevent of the event expressing the desired property, 
i.e., we show that whenever the coin event is satisfied, the desired property is satisfied as 
well. 

3. We use the lower bound on the probability of the coin event to obtain a lower bound on 
the probability of the desired property. 

Example 6.1.1 (Coin lemmas and the toy resource allocation protocol) Let us con- 
sider the toy resource allocation protocol of Chapter 5 again. One of the coin lemmas of 
this chapter states that if we fix any two separate coin flips (flipping of different coins) and 
we consider the event where the two coin flips yield different outcomes whenever they both 
occur, then, no matter how the nondeterminism is resolved, the considered event is satisfied 
with probability at least 1/2. On the other hand, if the first coin flip of Mi after the first coin 
flip of M2 is different from the last coin flip of Mi before the first time M\ checks its resource 
after flipping, then M\ succeeds in getting its resource. Thus, whenever the property above can 
be expressed as a coin event in a form suitable to the coin lemma above, we are guaranteed that 
M\ eventually gets its resource with probability at least 1/2. It turns out that an adversary 
must be fair, oblivious and deterministic in order to be able to define the desired coin event (cf. 
Section 6.6). Our results about deterministic and randomized adversaries (Proposition 5.7.11) 
can then be used to remove the constraint that an adversary is deterministic. ■ 

We present a large collection of coin lemmas, and we illustrate their use via two main examples: 
Section 6.3 proves the correctness of the randomized Dining Philosophers algorithm of Lehmann 
and Rabin [LR81], and Section 6.5 proves the correctness of the randomized algorithm of Ben- 
Or for agreement in asynchronous networks in the presence of stopping faults [B083]. At the 
end of the chapter we hint at another technique, called the partition technique, that departs 
considerably from the coin lemmas and that is necessary to prove stronger claims about the toy 
resource allocation protocol. We leave to further work a deeper study of this other technique. 

6.2 Some Simple Coin Lemmas 

In this section we present some simple coin lemmas where we use actions to identify the random 
draws of interest. Specifically, we study the following coin lemmas. 

1. First occurrence of an action. 

In this coin lemma we consider an action a and a set of states U, and we study the 
probability that either action a does not occur or the first occurrence of action a leads to 
a state of U . We show that this probability is at least the infimum of the probability of 
reaching a state of U over all the transitions of M that are labeled with action a. 
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As an example, action a can identify the process of flipping a fair coin and U can identify 
those states that are reached if the coin flip yields head. Then the coin lemma says that 
no matter how the nondeterminism is resolved the probability that either the coin is not 
flipped or the coin is flipped and yields head is at least 1/2. 

Observe that in the definition of the coin event we allow for those executions where no 
coin is flipped. One reason for this choice is to avoid trivial lower bounds due to the fact 
that a generic adversary can always decide not to schedule any transition. Another reason 
is that generally a randomized algorithm is structured so that that if no coin is flipped 
then progress is guaranteed with certainty. Alternatively, a randomized algorithm can be 
structured so that under any valid adversary some coin is flipped. In both cases it is of 
absolute importance to be aware of the existence of executions where no coin is flipped. 
Overlooking those executions is a common source of mistakes. 

2. First occurrence of an action among many. 

In this coin lemma we consider several pairs (a,-, [/,-) of actions and sets of states, and we 
study the probability that either none of the a 8 's occur or the action aj that occurs first 
leads to a state of Uj. We show that, if for each i pt is the lower bound given for (a,-, [/,-) 
by the coin lemma of 1, then the probability mentioned above is at least the minimum of 
the p 8 's. 

As an example, consider n processes that run in parallel, and suppose that each process 
can flip a fair coin. Then, the probability that either no process flips a coin or that the 
first process that flips a coin obtains head is at least 1/2. 

3. I-th occurrence of an action among many. 

In this coin lemma we consider the coin event of 2 with the difference that we consider 
the i occurrence of an action rather than the first occurrence. The lower bound on the 
probability of this event is the same as the lower bound on the probability of the event 
of 2. 

4. Conjunction of separate coin events. 

In this coin lemma we consider the conjunction of several coin events of the kind of 3. We 
show that if each one of the coin events involves disjoint occurrences of actions, then the 
lower bound on the probability of the conjunction is the product of the lower bounds on 
the probability of each of the involved coin events. 

As an example, consider n processes that run in parallel, and suppose that each process 
can flip a fair coin. For each i let X{ be either head or tail. Then, the probability that for 
each process i either no coin is flipped or the first coin that is flipped yields X{ is at least 

l/2 n . 

Some more general and complex coin lemmas are presented in Section 6.4; several other coin 
lemmas are likely to be derived in the future. Before presenting the simple coin lemmas in full 
detail we give just a rough idea of the coin lemmas of Section 6.4. 

5. Conjunction of separate coin events with multiple outcomes. 
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In this coin lemma we consider again the conjunction of several coin events that involve 
disjoint occurrences of actions. However we allow more freedom. First of all an action is 
paired with more than one set of states, thus allowing the observation of more than one 
outcome; second, we allow for multiple joint observations. 

As an example, the coin lemma says that if n processes run in parallel and each one of 
them can flip a coin, then the probability that at least half of the processes either do not 
flip a coin or flip head is at least 1/2. Similarly, if each process can roll a dice, then the 
probability that if process 1 rolls 1 then the other processes do not roll a number different 
from 1 is at least (l/6) n + 5/6, which is essentially the probability of rolling n dices and 
that either all processes give 1 or process 1 does not give 1. 

6. A generalized coin lemma. 

In this coin lemma we generalize the idea of 5, but this time we do not use actions to 
identify the random draws of interest. The reader is referred to Section 6.4.2 for further 
details. 

6.2.1 First Occurrence of an Action 

Let M be a probabilistic automaton, and let (a, U) be a pair consisting of an action of M and 
a set of states of M. Let FIRST(a, U) be a function that applied to a probabilistic execution 
fragment H of M returns the set of executions a of Q,jj such that either a does not occur in 
a>q^ , or a occurs in a>q^ and the state reached after the first occurrence of a is a state of U. 

It is simple to check that FIRST(a, U) is an event schema since, for each probabilistic 
execution fragment H of M, the complement of FIRST(a, U){H) is the set of executions a of 
£ljj such that action a occurs in a>q^ , and the state reached after the first occurrence of a is 
not a state of U . This set is expressible as a union of cones, and thus it is an event. 

The event schema FIRST(a, U) identifies the first random draw associated with action a 
that occurs in a probabilistic execution fragment H , and requires the outcome of the random 
draw to be in a specific range, namely in U . The intuition behind the use of such a coin event, 
is that a system performs well if the outcome of the first random draw involving a is in U . 
From the definition of FIRST(a, U), we assume also that the system performs well whenever a 
does not occur at all. Thus, if an adversary has the possibility not to schedule a, then it has a 
better chance to degrade the performance of a system by scheduling a. 

The following lemma provides a lower bound to the probability of FIRST (a, U). Informally, 
it states that if whenever there is a transition of M that involves action a the occurrence of a 
implies that a state of U is reached with probability at least p, then p is a lower bound on the 
probability of FIRST (a, U). 

Lemma 6.2.1 Let M be a probabilistic automaton, and let (a, U) be a pair consisting of an 
action of M and a set of states of M . Let p be a real number between and 1 such that for 
each transition (s,V) of M where P[a] > 0, P[?7|a] > p. Then, for each probabilistic execution 
fragment H of M , P H [FIRST(a, U)(H)] > p. 

Proof. For convenience denote FIRST(a, U){H) by E, and for each state q of H , denote by 
£l(q,U) the set {(a, g') £ fl^ \ Istate(q') £ U}. Let be the set of states q of H such that 
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action a does not occur in q>q$ , and P q [a] > 0. Then, 

Ph[E] = J2 E _?H[C q ]P q H [(«,</)]• (6.1) 

96© (a,g')efi(g,(7) 

By expressing P^[(a^ q')] as a conditional probability and rearranging the expression, we obtain 

P H [E] = J2PH[C q ]P q H [a]l J2 if [(«,?>]]. (6-2) 

see \(a, g ')efi(g,c/) / 

From the definition of a probabilistic execution fragment and the definition of £l(q, U), for each 
element q of there is a combined transition tr = J2iPi^ r i °f M such that £r;? = q" tr and 

Y^ pffrc >\\ i p r77i i p tr[Una] J2 t P t PtrA Una ] ( , Q x 

£ P, [«,. , )!«] = P„[VW\ = -p^- = . (6.3) 

(a, q ')&l(q,U) 

By multiplying and dividing each i summand of the enumerator by P ir , 8 [a], using the hypoth- 
esis of the lemma, i.e., for each i Pt ri [U fl a] < (1 — p), and simplifying algebraically, from (6.3) 
we obtain 

£ _P q H [(a,q')\a]<(l-p). (6.4) 

By using (6.4) in (6.2) we obtain 

Ph[E] < (1-p) (j2 P »lC q ]P q H [a]\ • (6.5) 

Furthermore, the subexpression J2 qe @ Ph [C q ]P q [a] is the probability that a occurs in if, which 
is at most 1. Thus, 

P H [E]<(l-p). (6.6) 

This completes the proof. ■ 

6.2.2 First Occurrence of an Action among Many 

The event schema FIRST(a, U) can be generalized to account for the first action that occurs 
among several possible ones. Let M be a probabilistic automaton, and let (ai, U\), . . . , (a n , U n ) 
be pairs consisting of an action of M and a set of states of M such that the actions a 8 - are 
all distinct. Then define FIRST((cii, U\), . . . , (a n , U n )) to be the function that applied to a 
probabilistic execution fragment H of M returns the set of executions a of S7# such that either 
none of the a 8 's occurs in a>q^ , or some of the a 8 's occur in a>q^ , and if a 8 - is the first of those 
actions that occurs, then the state reached after the first occurrence of a 8 - is a state of Ui. 

It is simple again to check that FIRST((cii, U\), . . . , (a n , U n )) is an event schema since, for 
each probabilistic execution fragment H , the complement of FIRST((cii, U\), . . . , (a n , U n )){H) 
can be expressed as a union of cones. 

Lemma 6.2.1 extends to this case. 
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Lemma 6.2.2 Let M be a probabilistic automaton, and let (ai, U\), . . ., (a n , U n ) be pairs con- 
sisting of an action of M and a set of states of M such that the actions ai are all distinct. Let 
{Pi}i=i,...,n be a collection of real numbers between and 1 such that for each i, 1 < i < n, 
and each transition (s,P) of M where P[ai\ > 0, P[P|a 8 -] > pi. Then, for each probabilistic 
execution fragment H of M , Ph[FIRST((o,i,Ui), . . . ,(a n ,U n ))(H)]> min(pi, . . . ,p n ). 

Proof. Let V be {ai, . . .,a n }, and let p be the minimum of {pi, . . -,p n }- For convenience, 
denote FIRST((ai, Pi), . . ., (a n , U n ))(H) by P, and for each state q of P, denote by S7(g, P) 
the set Ujgn ... jn }{( a «? <?') £ ^q I Istate(q') £ Ui}. Then, for each transition (q,V?) of P such 
that PH[V] > 0, 

P q H [£l(q,E)\V}<(l-p). (6.7) 

To prove (6.7), let, for each i = 1, . . . , ra, 0(g, a;, Ui) denote the set {(a;, q') G Q q | Istate(q') £ 
Ui}. Then, 

P q H [£l(q,E)\V}= Y, P?mq,ai,Ui)\V]. (6.8) 

ie{l,...,n} 

By using conditional probabilities, Equation (6.8) can be rewritten into 

P q H [£l(q,E)\V} = Y, PqVMmP^Mq.a^Ui)^]. (6.9) 

ie{l,...,n} 

Following the same argument as in the proof of Lemma 6.2.1, for each i, P^ [0(g, a;, Pi)|ai] < 
(1 — p); moreover, ^ 8 - P^ [a 8 - 1 F] = 1. Thus, (6.7) follows directly. 

The rest of the proof follows te lines of the proof of Lemma 6.2.1. Let be the set of states 
q of P such that no action of V occurs in q>q$ , and P^ [V] > 0. Then, 

Ph[E] =J2 E _ PH[C q ]P q H [(a,q')]. (6.10) 

1 e& {a,q')eQ{q,E) 

By expressing P^^a, q')] as a conditional probability and rearranging the expression, we obtain 

Ph[E] = J2 P H [C q ]P q H [V] J2 _ P?iM)\V] I • (6-11) 

see \{a,q')en{q,E) 

The subexpression J2t a ')cq( e) Pq J [(. a i < l')\V] 1S Pq^i^il^ E)\V], which is less than or equal to 
(1 — p) from (6.7). Thus, 

Ph[E] < (1-p) (j2 P H[C q ]P q H [V]j • (6.12) 

Furthermore, the subexpression J2 q e@ PH[C q ]P q [V] is the probability that an action from V 
occurs in P , which is at most 1. Thus, 

P H [E]<(l-p). (6.13) 

This completes the proof. ■ 
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6.2.3 I-th Occurrence of an Action among Many 

In the definition of FIRST we have considered the first action among a given set that occurs 
in a probabilistic execution fragment P. However, the results for FIRST are valid also if 
we consider the i th occurrence of an action instead of the first occurrence. This observation 
suggests a new more general event schema. 

Let M be a probabilistic automaton, and let (a\, Pi), . . . , (a n , U n ) be pairs consisting of 
an action of M and a set of states of M such that the actions a 8 - are all distinct. Then 
define OCC(i, (a\, Pi), . . . , (a n , U n )) to be the function that applied to a probabilistic execution 
fragment P of M returns the set of executions a of Q,jj such that either there are less than i 
occurrences of actions from {a\, . . . , a n } in a>q^ , or there are at least i occurrences of actions 
from {ai, . . .,a n }, and, if cij is the action that occurs as the i one, then the state reached 
after its occurrence is a state of P 8 -. 

Since in the proof of Lemma 6.2.2 we never use the fact that it is the first occurrence of an 
action that is considered, Lemma 6.2.2 carries over to the i occurrence trivially. 

Lemma 6.2.3 Let M be a probabilistic automaton, and let (a\, Pi), . . ., (a n , U n ) be pairs con- 
sisting of an action of M and a set of states of M such that the actions ai are all distinct. Let 
{Pj}j=i,...,n be a collection of real numbers between and 1 such that for each j £ {1, . . . ,n} 
and each transition (s,V) of M where P[a,j] > 0, P[P|aj] > pj. Then, for each probabilistic 
execution fragment P of M , Ph[OCC(i, (a\, Pi), . . . , (a n , U n ))(H)] > min(pi, . . .,p n ). ■ 

6.2.4 Conjunction of Separate Coin Events 

In this section we study what happens if we consider several events of the kind OCC together. 
In order to simplify the notation, we consider only event schemas of the kind OCC(i, (a, U)) 
since, as we have seen in the proof of Lemma 6.2.2, the case with multiple actions can be 
reduced to the case with a single action. 

The lemma that we prove states that if we consider several separate coin events, i.e., coin 
events that involve different random draws, each one with its own lower bound, then the lower 
bound of their conjunction is the product of the lower bounds. In other words, an adversary 
can introduce dependencies by increasing the probability of the conjunction of events, but it 
can never decrease the probability below the value that we would get by considering all the 
events to be independent. 

Lemma 6.2.4 Let M be a probabilistic automaton, and let (k\, a\, U\), . . ., (k n , a n , U n ) be a 
collection of triplets consisting of a natural number, an action of M and a set of states of 
M , such that the pairs (A; 8 ,a 8 ) are all distinct. Let {pj}j=\ ) ... ;n be a collection of real num- 
bers between and 1 such that for each j £ {l,...,ra} and each transition (s,V) of M 
where P[a,j] > 0, P[[7|aj] > pj. Then, for each probabilistic execution fragment H of M, 

Pnioccih, ( ai , Pi))(P) n • • • n occ(k n , (a n , u n ))(H)} > Pl --- Pn . 

Proof. For each / C {l,...,ra}, denote a generic event schema n 8e /OCC(A; 8 ', (a 8 , P 8 )) by ej. 
For each i = l,...,ra and each state q of P, denote by 0(g, i, [/,-) the set {(a 8 ,g') £ 0^ | 
Istate(q') £ U{\ of pairs where a 8 - occurs and Ui is reached, and denote by Cl(q,i,Ui) the set 
{(ai, q') £ 0^ | Istate(q') £ Ui} of pairs where a 8 - occurs and Ui is not reached. For each action 
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a and each state q of H , let a(q) denote the number of occurrences of action a in q>q$ . For 
each i = 1, . . . , n, let 8 be the set of states q of H such that each action a?, 1 < j < n occurs 
less than kj times in q>q$ , action a 8 - occurs &,- — 1 times in q>q$ , and Pg [«;] > 0. For each 
i = 1, . . . , n and each state q of H such that a 8 (g) < &,-, let OCC(ki, (a,-, Ui))>q denote the event 
schema OCC(k{ — cii(q), (a,-, P;)). Finally, for each / C {1, . . . , n} and each suitable state q of 
if, let ei>q denote the event schema P\i£iOCC(ki, (a,-, Ui))>q. 

We prove the lemma by induction on n. If ra = 1, then the result follows directly from 
Lemma 6.2.1. Otherwise, 

PH[ei,..., n (B)] = E E P »lC q ] E P?\M) 

ie{i,...,n} gee, \ \ ( at , q ')eQ{ q ,t,u t ) 



E 



Pf [(a,-,?')]^> g '[e{i,...,,--i,,-+i,..., n }>?'(^>?')] • (6-14) 



The first summand of Expression (6.14) expresses the probability that action a 8 - occurs from q 
and leads to a state not in [/,-; the second summand expresses the probability that a 8 - occurs, leads 
to a state of [/,-, and from the reached state something happen so that the resulting execution 
is not in ei v .. jn (Pf). From induction, and by using conditional probabilities, we obtain 

PH[ei,..., n (B)] < E E P H[C q ]P q H [a t ]([ £ if[(a,- >9 >,- 

ie{i,...,n} gee, y \(a 8 , g ')efi(g,«,c/0 

+ E P f [(»«■> ?')l«.-])(l " Pi • • -K-iK+i • • -Pn) • (6.15) 

\(a 8 ,g')efi(g, 8 ,C/0 // 

Let, for each i and each q, pi jQ = P^ [S7(g,i, P 8 )|a 8 ]. Then, (6.15) becomes 



Pff[ei,...,.(P)] 

^ E E P ^[^] P fN((l-P^) + (l-Pi---P 8 -iP«+i---PnK g ), (6.16) 

ie{l,...,n}5fG0i 

which becomes 

P H [ei,...,n(H)] < E E P H[C q ]P? N(l - Pi • • -Pi-lPi^Pi + l ■ ■ -p n ) (6.17) 

ie{i,...,n} gee, 

after simple algebraic simplifications. Using the same argument as in the proof of Lemma 6.2.1, 
for each i and each q, pt tq > pj. Thus, 

P H [ei,...,n(H)} < E E PH[C q ]P?[ai](l- Pl ---p n ). (6.18) 

ie{i,...,n} gee, 

Finally, observe that J2ie{i ...n}J2 q e@, PH[C q ]Pg[ai] is the probability that for some i action 
di occurs at least k{ times. Thus, 



PH[ei,...,n( H )]<( 1 -Pi---Pn)- (6-19) 

This completes the proof. ■ 
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Figure 6-1: The Dining Philosopher problem with 6 philosophers. 

6.3 Example: Randomized Dining Philosophers 

In this section we apply the methodology presented so far to prove the correctness of the Ran- 
domized Dining Philosophers algorithm of Lehmann and Rabin [LR81]. The proof is structured 
in two levels. The high level proof consists of a collection of progress statements that are con- 
catenated together; the low level proof consists of the proofs of the statements of the high level 
proof. The low level proof is based on the coin lemmas. 

6.3.1 The Problem 

There are n philosophers sat at a round table. Each philosopher has a plate in from of him, a 
fork on its left, and a fork on its right. The left fork is shared with his left neighbor philosopher, 
and the right fork is shared with his right neighbor philosopher. At the center of the table there 
is a bowl full of spaghetti. Figure 6-1 illustrates the situation for n = 6. Each philosopher 
goes repeatedly through phases where he is thinking and where he is eating. However, each 
philosopher needs both of its forks in order to eat. The problem is the following: 

"What procedure should each philosopher follow to get his forks and to put them 
down in order to make sure that every philosopher that is hungry will eventually be 
able to eatV 

A simpler problem is the following. 

" What procedure should each philosopher follow to get his forks and to put them down 
in order to make sure that whenever somebody is hungry somebody will eventually 
be able to eatV 

The second problem is simpler than the first problem since it allows for some philosopher 
to starve. It is known from [LR81] that there is no symmetric solution even for the simple 
dining philosophers problem, i.e., there is no deterministic solution for the dining philosophers 
problem where each philosopher follows exactly the same protocol; some mechanism to break 
the symmetry is necessary. In the algorithm of Lehmann and Rabin each philosopher follows 
exactly the same protocol and randomness is used to break the symmetry. 
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0. 


try 


1. 


< Ui <— random> ** 


2. 


< if Res(j u .) = free then 




Res (iiUj ) := taken 




else goto 2. > 


3. 


< if Res( 8 , OM)( „ 8 )) = free then 




R es (i,o PP (ui)) '■= taken; 




goto 5. > 


4. 


< Res (w) := free; goto l.> 


5. 


crit 




** Critical Section ** 


6. 


exit 


7. 


< Ui <— left or right 




Res( ii0 j,j,( Uj )) := free > 


8. 


< Res (iiUj ) := free > 


9. 


rem 



Shared variables: Resj £ {free, taken}, j = 1, . . . , ra, initially free. 
Local variables: U{ £ {left, right}, i = 1, . . . ,n 
Code for process i: 

** beginning of Trying Section ** 
choose left or right with equal probability ** 

** pick up first resource ** 

** pick up second resource ** 

** put down first resource ** 
** end of Trying Section ** 

** beginning of Exit Section ** 

** nondeterministic choice ** 

** put down first resources ** 

** put down second resources ** 

** end of Exit Section ** 



Figure 6-2: The Lehmann- Rabin algorithm. The operations between angular brackets are 
performed atomically. 

6.3.2 The Algorithm 

Each hungry philosopher proceeds according to the following protocol. 

1. Flip a fair coin to choose between the left and the right fork. 

2. Wait for the chosen fork to become free and get it. 

3. Try to get the second fork: 

if it is free, then get it; 

if it is taken, then put down the first fork and go to 1. 

4. Eat. 

Each philosopher that has terminated to eat puts down his forks one at a time. The intuition 
behind the use of randomness is that the actual protocol used by each philosopher is determined 
by an infinite sequence of random coin flips. Thus, with probability 1 each philosopher follows 
a different protocol. 

Figure 6-2 gives a more precise representation of the protocol, using a terminology that 
is closer to computer science; thus, a philosopher is called a process, and a fork is called a 
resource. A philosopher who is thinking is said to be in its reminder region; a philosopher 
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Figure 6-3: Numbering processes and resources in the Dining Philosophers problem. 

who is eating is said to be in its critical region; a philosopher who is trying to get its forks is 
said to be in its trying region; and a philosopher who is putting down its forks is said to be in 
its exit region. The n resources (forks) are represented by n shared variables Resi, . . . ,Res n , 
each of which can assume values in {free, taken}. Each process (philosopher) i ignores its 
own name and the names of its adjacent resources. However, each process i is able to refer 
to its adjacent resources by relative names: Res(, left \ is the resource located to the left, and 
R- es (i, right) i s the resource to the right of i. Each process i has a private variable Ui, whose value 
is in {left, right}, which is used either to keep track of the resource that process i currently 
holds, or, if no resource is held, to keep track of the resource that process i is going to take 
next. For notational convenience we define an operator opp that complements the value of its 
argument, i.e., opp(right) = left and opp(left) = right. 

We now define a probabilistic automaton M that represents the evolution of n philosophers. 
We assume that process i + 1 is on the right of process i and that resource Res 8 - is between 
processes i and i + 1 (see Figure 6-3). We also identify labels modulo n so that, for instance, 
process n + 1 coincides with process 1. 

A state s of M is a tuple (X\, . . . , X n , Resi, . . . , Res n ) containing the local state X{ of each 
process i, and the value of each resource Res 8 -. Each local state X{ is a pair (pc^Ui) consisting 
of a program counter pc i and the local variable U{. The program counter of each process keeps 
track of the current instruction in the code of Figure 6-2. Rather than representing the value 
of the program counter with a number, we use a more suggestive notation which is explained 
in Table 6.1. Also, the execution of each instruction is represented by an action. Actions try 4 -, 
critj-, rem;, exit; are external; all the other actions are internal. 

The start state of M assigns the value free to all the shared variables Res 8 -, the value R to 
each program counter pc,-, and an arbitrary value to each variable U{. The transition relation 
of M is derived directly from Figure 6-2. For example, for each state where pc i = F there is 
an internal transition labeled with flip, that changes pc i into W and assigns left to U{ with 
probability 1/2 and right to U{ with probability 1/2; from each state where X{ = (IF, left) 
there is a transition labeled with wait; that does not change the state if Res( 8]left \ = taken, 
and changes pc i into S and Res( 8]left ) into taken if Res( 8]left ) = free; for each state where 
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Nr. pc i Action Informal meaning 






R 


tr y 8 


Reminder region 


1 


F 


flip. 


Ready to Flip 


2 


W 


wait; 


Waiting for first resource 


3 


s 


second; 


Checking for Second resource 


4 


D 


drop, 


Dropping first resource 


5 


P 


crit; 


Pre-critical region 


6 


C 


exit; 


Critical region 


7 


E F 


dropf,- 


Exit: drop First resource 


8 


Es 


drops,- 


Exit: drop Second resource 


9 


Er 


rem; 


Exit: move to Reminder region 



Table 6.1: Program counter and action names for the Lehmann-Rabin algorithm. 

pc i = Ep there are two transitions labeled with action dropf,-: one transition sets m; to right 
and makes Res(,- left \ free, and the other transition sets m; to left makes Res(,- ri ht \ free. The 
two separate transitions correspond to a nondeterministic choice that is left to the adversary. 

The value of each pair X; can be represented concisely by the value of pc i and an arrow 
(to the left or to the right) which describes the value of m;. Thus, informally, a process i is in 
state S or D (resp. S or D) when i is in state S or D while holding its right (resp. left) 
resource; process i is in state W (resp. W) when i is waiting for its right (resp. left) resource 
to become free; process i is in state Es (resp. Es) when i is in its exit region and it is still 
holding its right (resp. left) resource. Sometimes we are interested in sets of pairs; for example, 
whenever pc i = F the value of m; is irrelevant. With the simple value of pc i we denote the set of 
the two pairs {(pc,-, left), (pc,-, right)}. Finally, with the symbol ^ we denote any pair where 
pc,- £ {W, S, D}. The arrow notation is used as before. 

For each state s = {X\, . . . , X n , Resi, . . . , Res n ) of M we denote X{ by X{(s) and Res 8 - by 
ReSj-(s). Also, for any set St of states of a process i, we denote by X{ £ St, or alternatively 
Xi = St the set of states s of M such that A 8 (s) £ St. Sometimes we abuse notation in the 
sense that we write expressions like Xi £ {F,D} with the meaning Xi £ F U D. Finally, we 
write Xi = E for X t = {E F , E s , E R }, and we write X t = F for X t £ {F, W, S, D,P}. 

6.3.3 The High Level Proof 

In this section we give the high level proof that the algorithm of Lehmann and Rabin guarantees 
progress, i.e., that from every state where some process is in its trying region, some process 
enters eventually its critical region with probability 1. We assume that each process that is 
ready to perform a transition is allowed eventually to do so: process i is ready to perform a 
transition whenever it enables an action different from try,- or exit;. Actions try,- and exit; 
are under the control of the user (a philosopher decides whether to eat or think), and hence, 
by assumption, under the control of the adversary. 

Formally, consider the probabilistic automaton M of Section 6.3.2. Define an extended 
execution a of M to be fair iff for each process i either a is finite and its last state enables 
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try j- or exit;, or a is infinite and either actions of process i occur infinitely many times in a 
or a = a\ ~ a.^ and all the states of a.^ enable either try,- or exit;. Define Fairadvs to be the 
set of adversaries A for M such that, for every finite execution fragment a of M the elements 
of Q p rexec(M,A,a) are extended fair execution fragments of M. Then Fairadvs is finite-history- 
insensitive: if A is an adversary of Fairadvs and q is a finite execution fragment of M, then it 
is easy to verify that the adversary A q such that 

A ( \ _ ) A(a>q) if q < a 
1 A(a) otherwise 

is an adversary of Fairadvs. Let rstates(M) denote the set of reachable states of M. Let 

T = {s e rstates(M) | 3 l X l {s) G {T}} 
denote the sets of reachable states of M where some process is in its trying region, and let 

C = {s G rstates(M) | 3 l X l {s) = C} 

denote the sets of reachable states of M where some process is in its critical region. We first 
show that 

T —^Fairadvs C, (6.20) 

1/8 

i.e., that, starting from any reachable state where some process is in its trying region, for all 
the adversaries of Fairadvs, some process enters its critical region eventually with probability at 
least 1/8. Note that (6.20) is satisfied trivially if some process is initially in its critical region. 
Our proof is divided into several phases, each one concerned with the property of making 
some partial progress toward C. The sets of states associated with the different phases are 
expressed in terms of T, 1ZT, J 7 , Q, V, and C. Here, 

UT = {seT\ ViXi(s) e {E R ,R,F}} 

is the set of states where at least one process is in its trying region and where no process is in 
its critical region or holds resources while being in its exit region. 

T = {seKT\ 3iXi(s) = F} 

is the set of states of 1ZT where some process is ready to flip a coin. 

V = {s e rstates(M) | 3 t X t (s) = P} 

is the sets of reachable states of M where some process is in its pre-critical region, i.e., where 
some process is ready to enter its critical region. The set Q is the most important for the 
analysis. To motivate the definition, we define the following notions. We say that a process i 
is committed if X{ £ {W, S}, and that a process i potentially controls Res 8 - (resp. Res 8 _i) if 
Xi G {W, S , D} (resp. X{ £ {W, S , D}). Informally said, a state in 1ZT is in Q if and only 
if there is a committed process whose second resource is not potentially controlled by another 
process. Such a process is called a good process. Formally, 

G = {seKT\ 3 t 

Xi(s) e {W,S} and X i+1 (s) G {E R ,R,F,#), or 

Xi(s) e {W,S} and X^s) G {E R ,R,F,f}} 
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Reaching a state of Q is a substantial progress toward reaching a state of C. Somehow, a good 
state is a place where the symmetry is broken. The progress statements of the proof are the 
following. 

T — > VJT U C (Proposition 6.3.3), 

VJT — > T U G U V (Proposition 6.3.16), 
T — > QUV (Proposition 6.3.15), 

1/2 V ' 

g — > V (Proposition 6.3.12), 

1/4 V ' 

V — ► C (Proposition 6.3.1). 

The first statement says that eventually every process in its exit region relinquishes its resources. 
In this way we avoid to deal with resources held by processes who do not want to enter the 
critical region. The second statement says that eventually either a good state is reached, or a 
place where some process is ready to flip its coin is reached. The flipping points are potential 
points where the symmetry is broken, and thus reaching a flipping point means progress. The 
third statement says that from a flipping point there is probability 1/2 to reach a good state. 
Finally, the fourth statement says that from a good state there is probability 1/4 to be ready 
to enter the critical region. By combining the statements above by means of Proposition 5.5.3 
and Theorem 5.5.2 we obtain 

T — >C, (6.21) 

1/8 ' V ; 

which is the property that was to be proven. Observe that once some process is in the trying 
region there is always some process in the trying region until some process reaches the critical 
region. Formally, M satisfies T Unless C. Thus, Proposition 5.5.6 applies, leading to 

T — >C. (6.22) 

6.3.4 The Low Level Proof 

In this section we prove the five progress statements used in Section 6.3.3. The proofs are 
detailed operational arguments. The main point to observe is that randomness is handled 
exclusively by the coin lemmas, and thus, any technique for the verification of ordinary automata 
could be applied as well. 

For the sake of clarity, we do not prove the relations in the order they were presented. 
Throughout the proof we abuse notation by writing expressions of the kind FIRST(f lip,, left) 
for the event schema FIRST(f lip,-, {s G states(M) \ Xi(s) = W}). We write also sentences of 
the form "If FIRST(f lip,-, left) then (f>" meaning that for each valid probabilistic execution 
fragment H, each element of FIRST(f lip,-, left)(if) satisfies (f>. 

Proposition 6.3.1 If some process is in P, then some process enters C, i.e., 

V — >C. 

l 

Proof. Let i be the process in P. Then, from the definition of Fairadvs, process i is scheduled 
eventually, and enters C . ■ 
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Lemma 6.3.2 If some process is in its Exit region, then it will eventually enter R. 

Proof. The process needs to perform two transitions to relinquish its two resources, and then 
one transition to send a rem message to the user. Every adversary of Fairadvs guarantees that 
those three transitions are performed eventually. ■ 

Proposition 6.3.3 T — >UTuC. 

Proof. From Lemma 6.3.2, every process that begins in Ep or Es relinquishes its resources. 
If no process begins in C or enters C in the meantime, then the state reached at this point is 
a state of 1ZT; otherwise, the starting state or the state reached when the first process enters 
C is a state of C. ■ 

We now turn to the proof of Q — ► V . The following lemmas form a detailed cases analysis 

1/4 

of the different situations that can arise in states of Q . Informally, each lemma shows that a 
specific coin event is a sub-event of the properties of reaching some other state. A preliminary 
lemma is an invariant of M, which guarantees that the resources are held by those processes 
who think to be holding them. 

Lemma 6.3.4 For each reachable state s of M and each i, 1 < i < n, Resi = taken iff 
Xi(s) £ { S , D , P, C , Ep, Es} or Xi + i(s) £ { S , D , P, C , Ep, Es}. Moreover, for each reachable 

state s of M and each i, 1 < i < n, it is not the case that Xi(s) £ {S , D ,P,C,Ep,Es} and 

Xi + i(s) £ { S , D ,P,C,Ep,Es}, i.e., only one process at a time can hold one resource. ■ 

Proof. The proof of this lemma is a standard proof of invariants. Simply verify that the two 
properties are true for the start states of M and are preserved by each transition of M. ■ 

Lemma 6.3.5 

1. Let Xi_i £ {Er,R,F} and Xi = W. If FIRST(f lip 8 _ 1 , left), then, eventually, either 
X l _ 1 = PorX l = S. *~ 

2. Let Xi_i = D and Xi = W. If FIRST (f lip 8 _ 1 , left), then, eventually, either X 8 _i = P 
or Xi = S . 

3. Let Xi_i = S and Xi = W. If FIRST(f lip 8 _ 1 , left), then, eventually, either X 8 _i = P 
or Xi = S . 

4- Let Xi_i = W and Xi = W. If FIRST (f lip 8 _ 1 , left), then, eventually, either X 8 _i = P 
or Xi = S . 

Proof. The four proofs start in the same way. Let s be a state of M satisfying the respective 
properties of items 1 or 2 or 3 or 4- Let A be an adversary of Fairadvs, and let a be an 
execution of £l vrexec (MAs},A) where the result of the first coin flip of process i — 1, if it occurs, 
is left. 



117 



1. By hypothesis and Lemma 6.3.4, i — 1 does not hold any resource at the beginning of a 
and has to obtain Res 8 _2 (its left resource) before pursuing Res 8 _i. From the definition 
of Fairadvs, i performs a transition eventually in a. If i — 1 does not hold Res 8 _i when 
i performs this transition, then i progresses into configuration S. If not, it must be the 
case that i — 1 succeeded in getting it in the meanwhile. But, in this case, since i — 1 flips 
left, ReSj-_i was the second resource needed by i — 1 and i — 1 therefore entered P. 

2. If Xi = S eventually, then we are done. Otherwise, process i — 1 performs a transition 
eventually. Let a = a\ ~ a.^ such that the last transition of a.\ is the first transition taken 
by process i — 1. Then Xi_i(fstate(a2)) = F and Xi(fstate(a2)) = W. Since process 
i — 1 did not flip any coin during a\ , from the finite-history-insensitivity of Fairadvs and 
Item 1 we conclude. 

3. If Xi = S eventually, then we are done. Otherwise, process i — 1 performs a transition 
eventually. Let a = a\ ~ a.^ such that the last transition of a.\ is the first transition taken 
by process i — 1. If Xi_i(fstate(a2)) = P then we are also done. Otherwise it must be 
the case that Xi_i(fstate(a2)) = D and Xi(fstate(a2)) = W. Since process i — 1 did not 
flip any coin during ai, from the finite-history-insensitivity of Fairadvs and Item 2 we 
conclude. 

4. If Xi = S eventually, then we are done. Otherwise, process i checks its left resource 
eventually and fails, process i — 1 gets its right resource before, and hence reaches at 
least state S . Let a = a\ ~ a.^ where the last transition of a.\ is the first transition of a 
that leads process i — 1 to state S . Then Xi_i(fstate(a2)) = S and Xi(fstate(a2)) = W. 
Since process i — 1 did not flip any coin during a\ , from the finite-history-insensitivity of 
Fairadvs and Item 3 we conclude. ■ 

Lemma 6.3.6 Assume that X 8 _i G {E R ,R,T} and Xi = W. If FIRST (f lip^, left), then, 
eventually, either X 8 _i = P or Xi = S . 

Proof. Follows directly from Lemma 6.3.5 after observing that X 8 _i £ {Er, R, T} is equivalent 
to X 8 _! G {E R , R, F, W, S, D,P}. m 

The next lemma is a useful tool for the proofs of Lemmas 6.3.8, 6.3.9, and 6.3.10. 

Lemma 6.3.7 Let Xi G {W, S} or X, G {E R ,R,F,D} with FIRST(f lip,-, left). Further- 
more, let X l+1 G {W, S} or X l+1 G {E R ,R,F,D} with FIRST(fliip i+1 , right). Then the 
first of the two processes i or i + 1 testing its second resource enters P after having performed 
this test (if this time ever comes). 

Proof. By Lemma 6.3.4 Res 8 - is free. Moreover, Res 8 - is the second resource needed by both i 
and i + 1. Whichever tests for it first gets it and enters P. ■ 

Lemma 6.3.8 If Xi = S and X 8 _|_i G {W, S} then, eventually, one of the two processes i or 
i + 1 enters P. The same result holds if Xi G {W, S } and X 8 _|_i = S . 
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Proof. Being in state S, process i tests its second resource eventually. An application of 
Lemma 6.3.7 finishes the proof. ■ 

Lemma 6.3.9 Let Xi = S and Xi + \ £ {E R ,R,F,D}. If FIRST (fliip i+1 , right), then, even- 
tually, one of the two processes i or i + 1 enters P. The same result holds if Xi £ {E R , R, F, D}, 
X i+1 = _£ and FIRST(f lip,-, left). 

Proof. Being in state S, process i tests its second resource eventually. An application of 
Lemma 6.3.7 finishes the proof. ■ 

Lemma 6.3.10 Assume that X;_i £ {E R ,R,T}, X % = W, and X l+1 £ {E R ,R,F,W, D}. If 
FIRST(f lip 8 _ 1 , left) and FIRST(f lij> i+1 , right), then eventually one of the three processes 
i — 1, i or i + 1 enters P. 

Proof. Let s be a state of M such that X 8 _i(s) £ {E R ,R,T}, Xi(s) = W, and X 8 _|_i(s) £ 
{Er,R,F,W, D}. Let A be an adversary of Fairadvs, and let a be an extended execution of 
Qprexec(M,{s},A) where the result of the first coin flip of process i — 1 is left and the result 
of the first coin flip of process i + 1 is right. By Lemma 6.3.6, eventually either process 
i — 1 reaches configuration P in a or process i reaches configuration S in a. If i — 1 reaches 

configuration P, then we are done. If not, then let a = a\ ~ a.^ such that lstate(a\) is the 
first state s' of a with Xi(s') = S . If i + 1 enters P before the end of ai, then we are done. 
Otherwise, Xi + i(fstate(a2)) is either in {W, S} or it is in {Er,R,F,D} and process i + 1 
has not flipped any coin yet in a. From the finite-history-insensitivity of Fairadvs we can then 
apply Lemma 6.3.7: eventually process i tests its second resource and by Lemma 6.3.7 process 
i enters P if process i + 1 did not check its second resource in the meantime. If process i + 1 
checks its second resource before process i does the same, then by Lemma 6.3.7 process i + 1 
enters P. ■ 

Lemma 6.3.11 Assume that X l+2 G {E R ,R,T}, X l+1 = W, and Xi £ {E R ,R,F,W, D}. If 
FIRST(f lip,, left) and F/i?5'T(f lip 8+2 , right), then eventually one of the three processes i, 
i + 1 or i + 2, enters P. 

Proof. The proof is analogous to the one of Lemma 6.3.10. This lemma is the symmetric case 
of Lemma 6.3.10. ■ 

Proposition 6.3.12 Starting from a global configuration in Q , then, with probability at least 
1/4, some process enters P eventually. Equivalently: 

G—*V. 

1/4 

Proof. Lemmas 6.3.8 and 6.3.9 jointly treat the case where Xi = S and A 8 _|_i £ {E R , R,F,i^} 
and the symmetric case where Xi £ {E R ,R,F, #} and A 8 _|_i = S ; Lemmas 6.3.10 and 6.3.11 
jointly treat the case where Xi = W and A 8 _|_i £ {Er, R, F,W, D} and the symmetric case 
where X % £ {E R , R, F,W,D} and X i+1 = W. 



119 



Specifically, each lemma shows that a compound event of the kind FIRST(f lip,, x) and 
FIRST(f lip •, y) leads to V . Each of the basic events FIRST^fliip}, x) has probability at least 
1/2. From Lemma 6.2.4 each of the compound events has probability at least 1/4. Thus the 
probability of reaching V eventually is at least 1/4. ■ 

We now turn to T — ► Q U V . The proof is divided in two parts and constitute the global 

1/2 

argument of the proof of progress, i.e., the argument that focuses on the whole system rather 
than on a couple of processes. 

Lemma 6.3.13 Start with a state s of T . If there exists a process i for which X 8 (s) = F and 
(X 8 _i,X 8 _|_i) 7^ (#,#), then, with probability at least 1/2 a state ofQUV is reached eventually. 

Proof. If s G Q U V, then the result is trivial. Let s be a state of T — (Q U V) and let i be such 
that Xi(s) = F and (X 8 _i,X 8 _|_i) ^ (#,#). Assume without loss of generality that X 8 _|_i ^ #, 

i.e., X 8 _|_i G {Er,R,F,#}. The case for X 8 _i ^ # is similar. Furthermore, we can assume 
that X 8 _|_i G {Er,R,F, D} since if X 8 _|_i G {W, S} then s is already in Q. We show that the 
event schema FIRSF((f lip^, left), (flip 8+1 , right)), which by Lemma 6.2.2 has probability 
at least 1/2, leads eventually to a state of Q U V . Let A be an adversary of Fairadvs, and let 
a be an extended execution of Q prexec (M,{s},A) where if process i flips before process i + 1 then 
process i flips left, and if process i + 1 flips before process i then process i + 1 flips right. 

Then, eventually, i performs one transition and reaches W. Let j G {i, i + 1} be the first of 
i and i + 1 that reaches W and let si be the state reached after the first time process j reaches 
W. If some process reached P in the meantime, then we are done. Otherwise there are two 
cases to consider. If j = i, then, flip, yields left and X 8 (si) = W whereas X 8 _|_i is (still) in 
{Er,R,F, D}. Therefore, si G Q. If j = i + 1, then flip 8+1 yields right and X 8 _|_i(si) = W 
whereas Xi(s\) is (still) F. Therefore, si G Q. ■ 

Lemma 6.3.14 Start with a state s of T '. If there exists a process i for which X 8 (s) = F and 
(X 8 _i(s),X 8 _|_i(s)) = (#,#). Fhen, with probability at least 1/2, a state of Q U V is reached 
eventually. 

Proof. The hypothesis can be summarized into the form (X 8 _i(s),X 8 (s),X 8 _|_i(s)) = (#,F,#). 
Since i — 1 and i + 1 point in different directions, by moving to the right of i + 1 there is a process 
k pointing to the left such that process k + 1 either points to the right or is in {Er,R,F,P}, 
i.e., X k (s) G {W, £, D] and X k+1 (s) G {E R , R, F, W, 5 , D, P}. 

If Xk(s) G {I'F, 5} and Xk+i(s) j^ P then s £ Q and we are done; if Xk+i(s) = P then 
s £ V and we are done. Thus, we can restrict our attention to the case where Xk(s) = D. 

We show that FIRST((f lij> k , left), (flip fc+1 , right)), which by Lemma 6.2.2 has proba- 
bility at least 1/2, leads eventually to Q U V . Let A be an adversary of Fairadvs, and let a 
be an extended execution of £l vrexec (MAs},A) where if process k flips before process k + 1 then 
process k flips left, and if process k + 1 flips before process k then process k + 1 flips right. 

Then, eventually, process k performs at least two transitions and hence goes to configuration 
W. Let j G {k, k + 1} be the first of k and k -\-l that reaches W and let si be the state reached 
after the first time process j reaches W. If some process reached P in the meantime, then we are 
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done. Otherwise, we distinguish two cases. If j = k, then, flipj. yields left and Xk(si) = W 
whereas Xk+\ is (still) in {Er,R,F, #}. Therefore, si G Q. If j = k + 1, then flip fc+1 yields 
right and Xk+i(si) = IF whereas Xfc(si) is (still) in {D,F}. Therefore, si G Q. ■ 

Proposition 6.3.15 Start with a state s of J 7 . Then, with probability at least 1/2, a state of 
Q U V is reached eventually. Equivalently: 

T — >guv. 

1/2 

Proof. The hypothesis of Lemmas 6.3.13 and 6.3.14 form a partition of T . ■ 

Finally, we prove 1ZT — > T U Q U V . 

Proposition 6.3.16 Starting from a state s of 1ZT , then a state of T U Q U V is reached 
eventually. Equivalently: 

KT — > T\jg\JV. 

Proof. Let s be a state of 1ZT. If s G JU(/UP, then we are trivially done. Suppose 
that s G" T U Q U V. Then in s each process is in {Er, R, W, S, D} and there exists at least 
process in {W,S,D}. Let A be an adversary of Fairadvs, and let a be an extended execution 

°f ^prexec(M,{s},A)- 

We first argue that eventually some process reaches a state of {S, D, F} in a. This is trivially 
true if in state s there is some process in {S,D}. If this is not the case, then all processes are 
either in Er or R or W. Eventually, some process in R or W performs a transition. If the 
first process not in Er performing a transition started in Er or R, then it reaches F and we 
are done; if the first process performing a transition is in W, then it reaches S since in s no 
resource is held. Once a process i is in {S, D, F}, then eventually process i reaches either state 
F or P, and we are done. ■ 

6.4 General Coin Lemmas 

The coin lemmas of Section 6.2 are sufficiently general to prove the correctness of the Random- 
ized Dining Philosophers algorithm of Lehmann and Rabin. However, there are several other 
coin events that are relevant for the analysis of distributed algorithms. For example, the toy 
resource allocation protocol that we used in Chapter 5 cannot be verified yet. In this section 
we present two general coin lemmas: the first one deals with multiple outcomes in a random 
draw; the second one gives a generalization of all the coin lemmas presented in the thesis. 
Unfortunately, generality and simplicity are usually incompatible: the two coin lemmas of this 
section are conceptually more complicated than those of Section 6.2. 

6.4.1 Conjunction of Separate Coin Events with Multiple Outcomes 

The coin lemma of Section 6.2.4 deals with the result of the intersection of several coin events. 
Thus, for example, if each coin event expresses the process of flipping a coin, then the coin 
lemma of Section 6.2.4 can be used to study the probability that all the coins yield head. 
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However, we may be interested in the probability that at least half of the coins yield head, 
or in the probability that exactly 5 coins yield head. The coin lemmas of Section 6.2 are not 
adequate. Suppose now that we use each coin event to express the process of rolling a dice. 
The coin events of Section 6.2 are not adequate again since they can deal only with binary 
outcomes: we can observe only whether a specific set U is reached or not. How can we express 
the event that for each number i between 1 and 6 there is at least one dice that rolls il 

In this section we define a coin event and prove a coin lemma that can deal with the scenarios 
outlined above. Let M be a probabilistic automaton, and let S be a set of n tuples {x\, . . . , x n }, 
where for each i, 1 < i < n, X{ is a tuple (a,-, Ui t \, . . . , Ui t k) consisting of an action of M and k 
pairwise disjoint sets of states of M. Let the actions a 8 - be all distinct. Let E be a set of tuples 
((l,ji), . . ., (n,j n )) where for each i, 1 < i < n, the value of ji is between 1 and k. For each 
extended execution a of M and each i, 1 < i < n, let 



Ui(a) 



{(i, 1), . . . , (i, k)} if (i{ does not occur 

{(i,j)} if di occurs and its first occurrence leads to Uij 

otherwise. 



Then define GFIRST(S, E) to be the function that associates with each probabilistic execution 
fragment H of M the set of extended executions a of £ljj such that E n (Ui(a>qQ ) X • • • X 
U k (a>qH)) ± 0. 

We illustrate the definition above by encoding the dice rolling example. In each tuple 
(a,-, Ui t i, . . ., Ui t k) o,i identifies the action of rolling the i th dice, k = 6, and for each j, Uij is 
the set of states where the i dice rolls j. The set E identifies the set of outcomes that are 
considered to be good. In the case of the dices E is the set of tuples ((1, ji), . . ., (n,j n )) where 
for each number / between 1 and 6 there is at least one i such that ji = I. The function Ui(a) 
checks whether the i th dice is rolled and identifies the outcome. If the dice is not rolled, then, 
we allow any outcome as a possible one; if the dice is rolled and hits Uij, then the outcome is 
(i,j); if the the dice is rolled and the outcome is not in any one of the sets t^j's, then there is 
no outcome (this case does not arise in our example). Then, an extended execution a of 0# 
is in the event GFIRST(S , E)(H) if at least one of the outcomes associated with a>qQ is an 
element of E, i.e., if by choosing the outcome of the dices that are not rolled in a>q^ all the 
six numbers appear as the outcome of some dice. 

Let p be the probability that by rolling n dices all the six numbers appear as the outcome 
of some dice. Then, the lemma below states that Pfj[GFIRST(S , E)(H)] > p for each H. 

Proposition 6.4.1 Let M be a probabilistic automaton. Let S be a set of n tuples {x\, . . . , x n } 
where for each i, 1 < i < n, X{ is a tuple (a,-, Ui t \, . . . , Ui t k) consisting of an action of M and k 
pairwise disjoint sets of states of M . Let the actions a,- be all distinct. Let E be a set of tuples 
((l,ji), . . ., (n,j n )) where for each i, 1 < i < n, the value of ji is between 1 and k. For each 
hj,^<i< n , 1 ^ J ' < k, let pi j be a real number between and 1 such that for each transition 
(s,V) of M where P[ai\ > 0, P[[/j-j|aj-] > pij, and let C be the collection of the Pijs. Let Pc[E] 
be the probability of the event E assuming that each experiment i is run independently, and 
that for each i a pair (i,j) is chosen with probability pij. Then, for each probabilistic execution 
fragment H of M , P H [GFIRST(S,E)(H)] > P C [E]. 

Proof. For each state q of H , each i £ {1, . . . , n}, and each j £ {1, . . . , k}, denote by Cl(q, Uij) 
the set {(a 8 ,g') £ 0^ | Istate(q') £ Uij} of pairs where a 8 - occurs and leads to a state of [/j-j, 
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and denote by 0(g, Ui) the set {(a 8 ,g') G 0? | Istate(q') £ Uj£/j-j} of pairs where a 8 - occurs and 
none of the Uijs is reached. For each i G {1, . . . , n}, let 8 be the set of states q of H such that 
no action aj, 1 < j ' < n, occurs in q>q$ , and -Pg [«;] > 0. 

We prove the lemma by induction on n. If ra = 1 then the result follows from Lemma 6.2.1 
(the event can be transformed into a new event with two outcomes); otherwise, 

P H [GFIRST{S,E){H)} = E E p H[C q ] E _ Pf [(«*,</) 

8 e{i, ...,«} gee, \ \(a 8 , g ')efi(gA) 

+ I E E PfKa.-^Ol^^'tGF/^r^^^^C^^')]) ) • (6-23) 

Vje{i,...,fc}(a 8 ,g')efi(g,c/ 8lJ ) / / 

where Si is obtained from S by removing the tuple (a,-, [/^i, . . . , £/,•,&), an d -EV 8J ) is the set of tu- 
ples ((l,j'i),...,(i-l,j 8 _i),(i! + l,j 8+ i),...,(ra,j n )) such that ((1, ji), . . ., (i-1, j 8 _i), (i, j), (i + 
1, Ji+i), • • ., (n, j n )) G -E. Let d be obtained from C by removing all the probabilities of the 
form pi j, 1 < j ' < k. Then, by induction, 



P H>q ,[GFIRST(S„ E {iij) )(H>q')] < (1 - Pc^m)])- (6.24) 

From the properties of conditional probabilities and the definition of C, 

Pc i [E(i, j )] = Pc[E\(i,j)]. (6.25) 

Thus, by using (6.24) and (6.25) in (6.23), and by expressing Pt^[(ai^ q')] as P^[ai]P^[(ai^ g')|a 8 ], 
we obtain 

P H [GFIRST(S,E)(H)]< E E p H[C q ]P? N E _ P f K^ON 

;e{i,...,n}<ze0i \\(a„,')esi(,,^) 

+ f E E if[(o,-,? , )|a,-](l-P c [i;|(i,i)]))). (6.26) 

For each i,j and g, let Pij, g be P/^[0(g, P;j)l a «]- Then, from (6.26), 

P H [GFIRST(S,E)(H)} < E E p ^] p f M 

ie{i,...,n}gee 8 

((1-P.M,, K,m) + f E Pi,J,^-Pc[E\(iJ)]))) , (6-27) 

V \ie{i,...,k} jj 

which becomes 



P H [GFIRST(S,E)(H) 



< E EWftl !" £ ^c[^|(i,j)Ki,J (6-28) 

;e{i,...,n}s(e0; \ je{i,...,fc} / 
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after some simple algebraic simplifications. Using the same argument as in the proof of 
Lemma 6.2.1, for each i,j and each q, Pi t j <q > Pij- Thus, 



P H [GFIRST(S,E)(H) 



< E E ^[C,]if [a,-] 1- £ Pc[E\(h3)]^ 3 • (6.29) 

i£{l,...,n} qe&i \ je{l,...,k} / 

Finally, observe that J2ieU n \ J2 q e& ^H [C q ]P q H [ai\ is the probability that some action a 8 - 
occurs, and observe that J2je{i ... k} Pc[E\(h j)]Pi,j = -fc [-£-]• Thus, 



P H [GFIRST(S, E)(H)] < 1 - P C [E] (6.30) 



6.4.2 A Generalized Coin Lemma 

All the coin lemmas that we have studied in this chapter share a common characteristic. Given 
a probabilistic execution fragment H , we identify n separate classes of random draws to observe. 
Each class can be observed at most once in every execution a of 0#, and if any class cannot 
be observed, then we allow for any arbitrary outcome. In this section we formalize this idea. 

Let if be a probabilistic execution fragment of a probabilistic automaton M. A coin-event 
specification for if is a collection C of tuples (q,X,X\, . . -,Xk) consisting of a state of if, a 
subset X of £1? , and m pairwise disjoint subsets of X, such that the following properties are 
satisfied: 

1. for each state q of H there is at most one tuple of C whose state is q; 

2. for each state q of H such that there exists a tuple of C with state q, there is no prefix q' 
of q such that there exists a tuple (q', X, X\, . . . , Xk) in C and a pair (a, q") in X where 
q" is a prefix of q. 

The set C is the object that identifies one of the classes of random draws to be observed. For 
each transition tr^ and each tuple (g, X, X\, . . . , Xk) of C, the set X identifies the part of tr^ 
that is relevant for C, and the sets X\, . . .,Xk identify some of the possible outcomes. The first 
requirement for C guarantees that there is at most one way to observe what happens from a 
state g of if , and the second requirement states that along every execution of 0# there is at 
most one place where C is observed. 

As an example, consider the observation of whether the first occurrence of an action a, 
which represents a coin flip, leads to head. Then C is the set of tuples (q,X,X\) where action 
a does not occur in q>q$ and P?[a\ > 0, X is the set of pairs of 0^ where action a occurs, 
and X\ is the set of pairs of X where the coin flips head. 

Let a be an extended execution of 0#, and let q be a state of H such that q < a. We say 
that C occurs in a at q iff there exists a tuple (q,X,X\, . . -,Xk) in C and a pair (a,q') in X 
such that q' < a. Moreover, if (a, q') £ Xj, we say that C occurs in a at q and leads to Xj. 

Two coin event specifications C\ and C'2 are said to be separate iff from every state q of 
H , if (q,Xi,Xi t i, . . .,Xi t k) is a tuple of C\ and (q, X2, -X^i, . . . ,X2 t k) is a tuple of C'2, then 
X\ fl X2 = 0. In other words, there is no interference between the observations of C\ and the 
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observations of Ci- Let S = {C\, . . .,C n } be a set of pairwise separate coin-event specifications. 
For notational convenience, for each i £ {1, . . . , n} and each state q of H such that there exists 
a tuple in C'i with state q, denote such tuple by (q,X qt i,X qt i t i, . . .,X qt i t k) 

Let E be a set of tuples ((1, j\), . . . , (n,j n )) where for each i, 1 < i < n, the value of ji is 
between 1 and k. For each extended execution a of 0# and each i, 1 < i < n, let 

{(i, 1), . . . , (i, A;)} if C; does not occur in a 
Ui(a) = < {(i,j)} if Ci occurs in a leading to X q< i t j 

otherwise. 

Then, define GCOIN(S , E)(H) to be the set of extended executions of Q,jj such that E n 
(tfi(a><^) X • • • X U k (a»q*)) ^ 0. 

Lemma 6.4.2 Le£ H be a probabilistic execution fragment of a probabilistic automaton M . Let 
S = {C\, . . . , C' n } be a set of separate coin-event specifications for H . For each i,j, 1 < i < n, 
1 < J ' < &> ^ Pi ,j ^ e a ret ^ number between an<i 1 swc/j that for each i £ {1, . . . , n} and each 
tuple (q, X q< i, X q< i tl , . .., X qthm ) ofC l} P q [X q ^^\X q ^} > p hr Let C be the collection of the p hJ 's. 
Let Pc[E] be the probability of the event E assuming that each experiment i is run independently, 
and for each i a pair (i,j) is chosen with probability pij. Then, Ph[GCOIN(S, E)(H)] > Pc[E]. 

Proof. For each state q of H and each i, 1 < i < n, if there exists a tuple in C'i with state q, 
then denote X q< i\ Ujg.n...^} X qt ij by X q ^. For each i, 1 < i < n, let 8 be the set of states q 
of H such that there exists a tuple with state q in C'i and no coin-event Cj, 1 < j < n, occurs 
in q>q$ ■ 

We prove the lemma by induction on n, using n = for the base case. For n = we assume 
that P[E] = 1 and that GCOIN(S , E)(H) = 0#. In this case the result is trivial. Otherwise, 

P H [GCOIN(S,E)(H)} = J2 E p H[C q ]\\ E_ P f iM) 

te{l,...,n}l£ & > \\(a,q')eX qti 

+ I E E P q H [(a,q , )]P H>ql [GCOIN(S>q',E ihj) )(H>q')]\ ] . (6.31) 

V6{l,..,t}(a,,')el, ] , j // 

where <St>g' is obtained from S by removing C'i and, for each j ^ i, by transforming the set Cj 
into { (<?><?', X>q', X\>q', . . . , Xk>q') | (g, X, X\, . . . , X^) £ Cj, q' < q}. Then, by induction, 



P H>q ,[GCOIN(S»q>,E {t)j) )(H»q>)} < (1 - P C , [%,)])• (6.32) 

From the properties of conditional probabilities and the definition of C, 

Pc i [E( i , j )] = Pc[E\(i,j)]. (6.33) 

Thus, by using (6.32) and (6.33) in (6.31), and expressing P^^a, q')] as P^[X qt i]P^[(a, q')\X qt i], 
we obtain 

P H [GCOIN(S,E)(H)]< J2 E P ^[^] P f[ X ^'] E_ P f[(«^OI^'] 

l e{i,...,n}gee t \\(a, g ')ex q ,i 

+ f E E P q H [(a,q')\X q Ml-Pc[E\(t,f)]))). (6.34) 

\je{i,...,k}(a,q')ex qtiJ ) ) 
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For each i,j and q, let Pij, g be P q [X q ^^\X q ^]. Then, from (6.34), 
P H [GCOIN(S,E)(H)] < £ E P H [C q ]P?[X qti ] 

ie{l,-,n} qe@, 

(1 - Pi,i, q Pi,k, q ) + E PiJ«( l ~ Pc[E\{i,j)}) , (6.35) 

V \j€{i,...,fc} // 

which becomes 



P H [GCOIN(S,E)(H)} 



< E Y, P HlC<,]P?lXi,j] I" E ^W,j)]^-, g (6-36) 

8 'e{i,..., n }gee 8 \ je{i,...,fc} / 

after some simple algebraic simplifications. From hypothesis, for each i,j and each q, Pi t j <q > 
Pij. Thus, 



P H [GCOIN(S,E)(H)} 



< E E ^[C,]if [X,,,-] 1- J] i'c^Kt.i)]^- • (6-37) 

ie{i,...,n}v£ @ i V j'e{i,...,fc} / 

Finally, observe that J2ie{i ...n} J2 q e@, PH[C q ]Pq[X qi i] is the probability that some d occurs, 
and observe that J2je{i,...,k} Pc[E\(i,j)]Pi,j = Pc[E]. Thus, 



P H [GCOIN(S,E)(H)] < 1 - P C [E] (6.38) 



6.5 Example: Randomized Agreement with Stopping Faults 

In this section we analyze the Randomized Agreement algorithm of Ben-Or [B083]. Its proof 
of correctness is an application of Lemma 6.4.2. The proof that we present in this section is not 
as detailed as the proof of the Dining Philosophers algorithm, but contains all the information 
necessary to fill in all the details, which we leave to the reader. 

6.5.1 The Problem 

Consider n asynchronous processes that communicate through a network of reliable channels 
(i.e., channels that deliver all the messages in the same order as they are received, and that 
never fail to deliver a message), and suppose that each process i starts with an initial value 
Vi G {0, 1}. Suppose that each process can broadcast a message to every other process in a 
single operation. Each process runs an algorithm that at some point may decide on one value 
of {0, 1}. Each process decides at most once. The algorithm should be designed so that the 
following properties are satisfied. 

1. Agreement: all the processes that decide choose the same value. 
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2. Validity: if all the processes have the same initial value v, then v is the only possible 
decision value. 

3. /-failure termination: if at most / processes fail, then all the non-failing processes 
decide a value. 

We assume that a process fails by stopping, i.e., by failing to send messages to other processes 
from some point on. Since the processes are asynchronous, no processes can distinguish a slow 
process from a failing process. 

Unfortunately, it is known from [FLP85] that there is no deterministic algorithm for asyn- 
chronous processes that solves the agreement problem and guarantees 1-failure termination. 
Here we present the randomized algorithm of Ben-Or [B083], which solves the agreement prob- 
lem with certainty, and guarantees /-failure termination with probability 1 whenever n > 3/. 

6.5.2 The Algorithm 

Each process i has local variables x, initially Vi, and y, initially null, and executes a series of 
stages numbered 1,2,.. ., each stage consisting of two rounds. Each process runs forever, even 
after it decides. At stage st > 1, process i does the following. 

1. Broadcast (first, st,v), where v is the current value of x, and then wait to obtain n — f 
messages of the form (first, st, *), where * stands for any value. If all the messages have 
the same value v, then set y := v, otherwise set y := null. 

2. Broadcast (second, st, v), where v is the current value of y, and then wait to obtain n — f 
messages of the form (second, st, *). There are three cases: 

(a) if all the messages have the same value v j^ null, then set x := v and perform a 
decide(v)i operation if no decision was made already; 

(b) if at least n — 2/ messages, but not all the messages, have the same value v j^ null, 
then set x := v without deciding (the assumption n > 3/ guarantees that there 
cannot be two different such values v); 

(c) otherwise, set a; to with probability 1/2 and to 1 with probability 1/2. 

The intuition behind the use of randomness is that at each stage, if a decision is not made yet, 
with probability at least 1/2™ all the processes that choose a value at random choose the same 
"good" value. Thus, with probability 1 there is eventually a stage where the processes that 
choose a value at random choose the same good value, and this leads to a decision. 

We now give an idea of the structure of the probabilistic automaton M that describes Ben- 
Or's algorithm. Each process i has the two variables x and y mentioned in the description 
of the algorithm, plus a queue nij for each process j that records the unprocessed messages 
received from process j, initially null, a stage counter st, initially 1, a program counter pc, 
and a boolean variable decided that is set to true iff process i has decided already. There 
is a channel C 8 j between every pair of processes. Each channel C 8 j is essentially a buffer 
like the buffer described in Chapter 3 (cf. Figure 3-1), whose inputs are actions of the form 
(first, st,v)i and (second, st,v)i, and whose outputs are actions of the form (first, st,v)ij and 
(second, st, f)«,j- To broadcast a message (first, st, v), process i performs the action (first, st, v)i. 
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A message (first, st, v) is received by process i from process j through the action (first, st, f)j,«- 
The definition of the transition relation of M is straightforward. 

6.5.3 The High Level Proof 

Agreement and validity are easy to prove and do not involve any probabilistic argument. 

Lemma 6.5.1 Ben-Or's algorithm satisfies the agreement and validity conditions. 

Proof. We start with validity. Suppose that all the processes start with the same value v. 
Then it is easy to see that every process that completes stage 1 decides on v in that stage. This 
is because the only value sent or received by any process in the first round is v, and thus the 
only value sent or received by any process in the second round is v, leading to the decision of v. 
For agreement, suppose that some process decides, and let process i be the first process 
that decides. Let v and st be the value decided by process i and the stage at which process 
i decides, respectively. Then it must be the case that process i receives n — f (second , st , v) 
messages. This implies that any other process j that completes stage st receives at least n — 2f 
(second , st , v) messages, since it hears from all but at most / of the processes that process i 
hears from. This means that process j cannot decide on a value different from v at stage st; 
moreover, process j sets x := v at stage st. Since this is true for all the processes that complete 
stage st, then an argument similar to the argument for validity shows that any process that 
completes stage st + 1 and does not decide in stage st decides v at stage st + 1. ■ 

The argument for /-failure termination involves probability. We assume that all the processes 
but at most / are scheduled infinitely many times. Thus, let f-fair be the set of adversaries for 
M such that for each probabilistic execution fragment H generated by an adversary of f-fair 
the set Q,jj contains only executions of M where at least n — f processes are scheduled infinitely 
many times. It is easy to check that f-fair is finite-history-insensitive. 

Let B be the set of reachable states of M; let T be the set of reachable states of M where 
no process has decided yet and there exists a value st and a number i such that process i 
received exactly n — f messages (first, st, *), and no other process has ever received more than 
n — f — 1 messages (first, st, *); finally, let O be the set of reachable states of M where at least 
one process has decided. 

It is easy to show that 

B-^.fairFUO. (6.39) 

Specifically, let a be an /-fair execution fragment of M starting from a reachable state s of M , 
and let st be the maximum value of the stages reached by each process in s. Then, stage st + 1 
is reached eventually in a, and thus there is a state s' in a where some process is the first one 
to receive n — f messages (first, st + 1, *). The state s' is a state of T U O. 
In Section 6.5.4 we show that 

T — >0. (6.40) 

1/2" v ' 

Thus, combining (6.39) and (6.40) with Theorem 5.5.2, and by using Proposition 5.5.6, we 
obtain 

B-^O. (6.41) 
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Finally, we need to show that in every /-fair execution where at least one process decides all 
the non-failing processes decide eventually. This is shown already in the second part of the 
proof of Lemma 6.5.1. 

6.5.4 The Low Level Proof 

In this section we prove the progress statement of (6.40) using the generalized coin lemma. 
Consider a state s of T , and let i be the process that has received n — f messages (first, st, v). 
Let A be an adversary of f-fair, and let H be prexec(M,A,s). 

For each j ', 1 < j ' < n, let Cj be the set of triplets (q, X, X\) where q is a state of H such that 
process j is at stage st in Istate(q) and there is a non-zero probability that process j chooses 
randomly between and 1 from q, X is the set of pairs of fl^ where process j performs a 
transition, and X\ is defined as follows. Let s' be Istate(q), and let v be a good value if at least 
/ + 1 of the messages (first, st, *) processed by process i have value v. We emphasize the word 
"processed" since, although each process can receive more that n — f messages (first, st, *), only 
n — f of those messages are used (processed). 

1. If is a good value, then let X\ be the set of pairs of X where process i chooses 0; 

2. if 1 is a good value and is not a good value, then let X\ be the set of pairs of X where 
process i chooses 1. 

Observe that in s' there is at least one good value, and at most two values; thus, Cj is well 
defined. It is easy to check that C\, . . . ,C n are separate coin event specifications; more- 
over, for each j, 1 < j < n, and each triplet (q,X,X\) of Cj, P^[X\\X] = 1/2. Let 
E = {((1, 1), (2, 1), . . . , (n, 1)}. From Lemma 6.4.2, P H [GCOIN((d, ..., C n ), E)(H)} > l/2 n . 

We are left with the proof that in each extended execution of GCOIN((C\, . . . , C n ), E)(H) 
all the non-faulty processes choose a value. More precisely, we show that the non-faulty pro- 
cesses complete stage st setting x to the same value v. Then, the second part of the proof of 
Lemma 6.5.1 can be used to show that all the non-faulty processes decide on v at the end of 
stage st + 1; in particular at least one process decides. We distinguish two cases. 

1. In s' there is exactly one good value v. 

In this case every other process receives at least one copy of v during the first round of 
stage st, and thus y is set either to v or to null. Therefore, v is the only value that 
a process chooses by a non-random assignment at the end of stage st. On the other 
hand, if a process j chooses a value at random at the end of stage st, the definition of Cj 
guarantees that the value chosen is v. Thus, every process that completes stage st sets 
x := v. 

2. In s' there are two good values. 

In this case every process receives at least one copy of and one copy of 1, and thus y 
is set to null. Therefore, each process chooses a value at random at the end of stage st. 
The definition of C\, . . . ,C n guarantees that every process that completes stage st sets 
x := 0. 
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6.6 Example: The Toy Resource Allocation Protocol 

Lemma 6.4.2 can be used also to prove formally that the toy resource allocation protocol of 
Section 5.1 guarantees that, under any deterministic fair oblivious adversary (cf. Example 5.6.2 
for the definition of a fair oblivious adversary), process Mi eventually gets a resource. This 
result can be extended to general oblivious adversaries by using the results about deterministic 
and randomized adversaries proved in Chapter 5 (cf. Proposition 5.7.11). 

Recall from Example 6.1.1 that we want to identify a coin event that expresses the following 
property: the first coin flip of Mi after the first coin flip of Mi is different from the last coin 
flip of M2 before the first time Mi checks its resource after flipping. In the rest of the section 
we specify two coin event specifications C\ and Ci- The specification C\ identifies the first coin 
flip of Mi after the first coin flip of M2, while the specification C'2 identifies the last coin flip of 
M2 before the first time Mi checks its resource after flipping. 

Let if be a probabilistic execution fragment, generated by a deterministic fair oblivious 
adversary, such that the first state of q$ is reachable in M. Let C\ be the set of tuples 
(q,X,Xi,X2) where 

1. q is a state of H such that M2 flips at least once in q>q$ , M\ does not flip in q>q$ after 
the first time M2 flips, and Mi flips from q, 

2. X is the set 0^", 

3. X\ is the set of pairs of X where Mi flips head, 

4. X2 is the set of pairs of X where Mi flips tail. 

Observe that C\ is a coin-event specification. Moreover, observe that for each tuple of C\, 
Pf \X X \X\ = 1/2 and Pf [X 2 \X] = 1/2. Let C 2 be the set of tuples (q,X,X 1 ,X 2 ) where 

1. q is a state of H such that either 

(a) Mi does not flip in q>q$ after M2 flips, M2 flips from q, and there exists a state 
q' > q such that M2 flips exactly once in q'\>q and Mi flips and checks its resource 
after flipping in q'>q, or 

(b) Mi flips and does not check its resource after the first flip of M2 in q>q$ , M2 flips 
from q, and there exists a state q' > q such that M2 flips exactly once in q'>q, Mi 
does not check its resource in q'>q, and Mi checks its resource from q' , 

2. X is the set 0^", 

3. X\ is the set of pairs of X where M2 flips head, 

4. X\ is the set of pairs of X where M2 flips tail. 

Informally, C'2 identifies the coin flip of M2 that precedes the point where Mi checks the 
resource determined by C\. Figure 6-4 illustrates graphically the two cases of the definition 
of C 2 . Observe that for each tuple of C 2 , Pf [X X \X\ = 1/2 and P^[X 2 \X] = 1/2. Since H is 
generated by an oblivious deterministic adversary, then it is easy to verify that C'2 is a coin-event 
specification. The important point is to verify that Condition 2 of the definition of a coin event 
is satisfied; this is the point where the fact that an adversary is oblivious and deterministic is 
used. 
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Figure 6-4: The definition of C'2 for the toy resource allocation protocol. 

Mjt 



H, . M 2-f 




MJ M >< 



//> 




Figure 6-5: How C'2 could not be a coin event specification. 

Example 6.6.1 (How C 2 could not be a coin event specification.) To give a rough idea 
of why Condition 2 does not fail, Figure 6-5 shows how Condition 2 could fail. Consider the 
execution of Hi that is marked with *, and denote it by a; denote by a' the other execution of 
Hi that appears in the figure. The unfilled circles mark the points where a coin event speci- 
fication is observed. By following a from left to right we observe C\ and then we observe Ci- 
The reason why we observe C'2 the first time is that along a' Mi tests its resource. However, 
continuing to follow a, we observe C'2 again because along a M2 tests its resource later. Using 
oblivious adversaries we are guaranteed that such a situation does not arise because if along a' 
Mi tests its resource before M2 flips again, then the same property holds along a. 

The probabilistic execution H2 of Figure 6-5 illustrates how Condition 2 can fail by using 
randomized schedulers. After Mi flips, the adversary chooses randomly whether to let Mi test 
its resource (higher filled circle) or to let M2 continue. ■ 

Let E be the set {((1, 1)(2,2)), ((1, 2), (2, 1))}, which expresses the fact that C\ and C'2 yield 
two different outcomes. It is easy to check that in every execution of GrC027V((Ci, C2), E)(H) 
Mi eventually gets one resource. Thus, from Lemma 6.4.2, the probability that Mi gets its 
resource in H is at least 1/4. Since if is a generic probabilistic execution fragment, then, under 
any deterministic fair oblivious adversary Mi gets a resource eventually with probability at 
least 1/4. Since the set of deterministic fair oblivious adversaries is finite-history-insensitive, 
Lemma 5.5.6 applies, and we conclude that under any deterministic fair oblivious adversary Mi 
gets a resource eventually with probability 1. 
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6.7 The Partition Technique 

Even though the coin lemmas can be used to prove the correctness of several nontrivial algo- 
rithms, two of which have been illustrated in this chapter, there are algorithms for which the 
coin lemmas do not seem to be suitable. One example of such an algorithm is the random- 
ized algorithm for maximal independent sets of Awerbuch, Cowen and Smith [ACS94]; another 
example is the toy resource allocation protocol again. 

Example 6.7.1 (The coin lemmas do not work always) In Section 6.6 we have shown 
that the toy resource allocation protocol guarantees progress against fair oblivious adversaries; 
however, in Example 5.6.2 we have stated that the toy resource allocation protocol guarantees 
progress also against adversaries that do not know only the outcome of those coins that have 
not been used yet. Such a result cannot be proved using the coin lemmas of this chapter be- 
cause situations like those outlined in Example 6.6.1 arise. For example, after the first time Mi 
flips, we could schedule Mi again and then schedule M\ to test its resource only if Mi gets the 
resource R\. 

Another way to obtain a situation where the coin lemmas of this chapter do not apply is to 
modify the second instruction of the resource allocation protocol as follows 

2. if the chosen resource is free, then get it, otherwise go back to 1 . ■ 

Example 6.7.1 shows us that some other techniques need to be developed; it is very likely that 
several new techniques will be discovered by analyzing other algorithms. In this section we hint 
at a proof technique that departs considerably from the coin lemmas and that is sufficiently 
powerful to deal with the toy resource allocation protocol. We illustrate the technique with an 
example. 

Example 6.7.2 (The partition technique) Let A be a generic fair adversary for the toy 
resource allocation protocol that does not know the outcome of those coin flips that have not 
been used yet, and let if be a probabilistic execution generated by A. Assume for simplicity that 
A is deterministic; the result for a generic adversary follows from Proposition 5.7.11. Consider 
an element of S7#, and consider the first point q where Mi flips a coin (cf. Figure 6-6). The 
coin flipping transition leads to two states qh and q t that are not distinguishable by A, which 
means that from qh and q t the adversary schedules the same process. If the process scheduled 
from qh and q t is M2, then the states reached from qh are in one-to-one correspondence with the 
states reached from q t , since they differ only in the value of the coin flipped by M\. Figure 6-6 
illustrates the case where Mi flips a coin. Furthermore, two corresponding states are reached 
with the same probability. The one-to-one correspondence between the states reached form qh 
and q t is maintained until M\ tests its chosen resource. 

Consider now a point where M\ tests its resource. Figure 6-6 illustrates four of these points, 
denoted by q t< i, qh,i, qt,2, an d qh,2- If M\ fails to obtain the resource, it means that Mi holds 
that resource at that point. However, Mi holds the same resource in the corresponding state 
via the one-to-one correspondence M2, while M\ tests the other resource. Thus, M\ succeeds 
in getting the chosen resource, (cf. states q t) \ and qh t i of Figure 6-6. 

The bottom line is that we have partitioned the states where Mi checks its resource in 
two sets, and we have shown that for each pair of corresponding states there is at least one 
state where Mi succeeds in getting a resource. In some cases, like for states q t< 2, and q^^ of 
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Figure 6-6: The partition technique. 

Figure 6-6, M\ succeeds in getting its resource from both the corresponding states (M2 does 
not hold any resource). Thus, M\ gets a resource with probability at least 1/2. ■ 

6.8 Discussion 

To our knowledge, no techniques similar to our coin lemmas or to our partition technique were 
proposed before; however, similar arguments appear in several informal analysis of randomized 
algorithms. The idea of reducing the analysis of a randomized algorithm to the analysis of an 
ordinary pure nondeterministic system was at the base of the qualitative analysis techniques 
described in Sections 2.5.1 and 2.5.2. Here we have been able to apply the same idea for a 
quantitative analysis of an algorithm. 

In this chapter we have focused mainly on how to apply a coin lemma for the verification of 
a randomized algorithm; once a good coin event is identified, the analysis is reduced to verify 
properties of a system that does not contain randomization. We have carried out this last part 
using detailed operational arguments, which can be error prone themselves. However, since the 
problem is reduced to the analysis of a non-randomized system, several existing techniques can 
be used to eliminate our operational arguments. In [PS95] Segala and Pogosyants show how 
such an analysis can be carried out formally and possibly mechanized. 
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Chapter 7 

Hierarchical Verication Trace 
Distributions 

7.1 Introduction 

So far we have defined a model to describe randomized concurrent and distributed systems, 
and we have shown how to study the properties of a system by means of a direct analysis of its 
structure. A specification is a set of properties that an implementation should satisfy, and an 
implementation is a probabilistic automaton that satisfies the desired properties. 

Another approach to the analysis of a system considers an automaton as a specification itself. 
Then, an abstract notion of observation is defined on automata, and an automaton is said to 
be an implementation of another automaton iff there is a specific relation, usually a preorder 
relation, between their abstract observations. Examples of observations are traces [Hoa85, LV91] 
(cf. Section 3.2.3), and failures [Hoa85, BHR84]; in these two cases implementation is expressed 
by set inclusion. 

7.1.1 Observational Semantics 

Formally, an automaton A is associated with a set Obs(A) of observations, and a preorder 
relation 1Z is defined over sets of observations (for example 1Z can be set inclusion). Then, an 
automaton A\ is said to implement another automaton A2, denoted by A\ C A 2 , iff Obs(A\) 1Z 
0bs(A2). The function Obs() is called an observational semantics , or alternatively a behavioral 
semantics] in the second case the observations are thought as the possible behaviors of an 
automaton. 

The methodology based on preorder relations is an instance of the hierarchical verification 
method: a specification, which is usually very abstract, can be refined successively into less 
abstract specifications, each one implementing the more abstract specification, till the actual 
implementation is obtained. Figure 7-1 gives an example of a specification that is refined two 
times to build the actual implementation. Of course it is implicitly assumed that the relevant 
properties of a system are only those that are preserved by the chosen implementation relation. 
Thus, given a relation, it is important to understand what properties it preserves. Coarse 
relations may not preserve all the relevant properties, but they are usually easy to verify, i.e., it 
is usually easy to establish whether such a relation holds; finer relations that preserve exactly the 
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Figure 7-2: Modular design. 

relevant properties are usually difficult to characterize and verify; other relations that preserve 
all the relevant properties and that are easy to verify are usually too fine, i.e., they distinguish 
too much. Some tradeoff is necessary. 

7.1.2 Substitutivity and Compositionality 

When the size of a problem becomes large, it is common to decompose the problem into simpler 
subproblems that are solved separately. Figure 7-2 gives an example. A large specification S is 
decomposed into several subcomponents Mi, . . . , M n that interact together to implement S. For 
example, a complex computer system can be described by the interaction of a central processor 
unit, a memory unit, and an Input/Output unit. Then, each subcomponent specification M 8 - is 
given to a development team that builds an implementation M[. Finally, the implementations 
are put together to build an actual implementation of S. This kind of approach is called modular 
design; however, in order to guarantee the soundness of modular design, we need to guarantee 
that an implementation works properly in every context where its specification works properly, 
i.e., our implementation relation must be preserved by parallel composition (i.e., it must be a 
precongruence) . This property is called substitutivity of a preorder relation, and constitutes one 
of the most important properties that an implementation relation should satisfy. 

A property that is strictly related to the substitutivity of C is called compositionality 
of ObsQ. That is, there is an operator || defined on pairs of sets of observations such that 
06s(Ai||A2) = 0bs(Ai)\\0bs(A2). Compositionality and substitutivity are used interchange- 
ably when talking informally about concurrent systems, and it is easy to get confused by the 
meanings of the two terms. To clarify every doubt, here is how the two concepts are related. 
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Theorem 7.1.1 Let Obs() be an observational semantics, 1Z be an equivalence relation over 
sets of observations, and let, for each set x of observations, [x]n be the equivalence class of 
x under 1Z. Let A\ = Ai iff Obs(A\) 1Z Obs(A2). Then the following two statements are 
equivalent. 

1. = is substitutive, i.e., if A\ = Ai then for each A3, Ai\\As = A2HA3; 

2. Obs() is compositional, i.e., there exists an operator || on equivalence classes of observa- 
tions such that [Obs(A 1 \\A 2 )]n = [Obs(A 1 )]n\\[Obs(A 1 )]n- ■ 

If 1Z is set equality, then we can remove the equivalence classes from the second statement 
since each set of observations is an equivalence class. The substitutivity of a preorder relation 
is stronger than the substitutivity of its kernel equivalence relation, since the direction of the 
inequality must be preserved under parallel composition. For this reason our primary concern 
in this chapter is the substitutivity of the implementation relation. 

7.1.3 The Objective of this Chapter 

In this chapter we study the simplest implementation relation based on observations, i.e., trace 
inclusion, and we extend the corresponding precongruence to the probabilistic framework. The 
trace preorder constitutes the basis for several other implementation relations and is known to 
preserve the safety properties of a system [AS85]. Roughly speaking, a safety property says that 
"something good holds forever" or that "something bad does not happen". The trace preorder 
is important for ordinary automata for its simplicity and for the availability of the simulation 
method [LT87, Jon91, LV91] (cf. Chapter 8), which provides several sufficient conditions for 
the trace preorder relation to hold. Other relations, based either on failures [Hoa85, BHR84] 
or on any other form of enriched traces, can be obtained by following the same methodology 
that we present here. 

In the probabilistic framework a trace is replaced by a trace distribution, where the trace 
distribution of a probabilistic execution fragment H is the distribution over traces induced by 
Vh-i the probability space associated with H. The trace distribution preorder is defined as 
inclusion of trace distributions. 

Unfortunately, the trace distribution preorder is not a precongruence (cf. Example 7.4.1), 
which in turn means that the observational semantics based on trace distributions is not com- 
positional. A standard approach in this case is to define the trace distribution precongruence 
as the coarsest precongruence that is contained in the trace distribution preorder; then, in 
order to have a compositional observational semantics that captures the trace distribution pre- 
congruence, an alternative, more operational and constructive characterization of the trace 
distribution precongruence is derived. We give an alternative characterization of the trace dis- 
tribution precongruence by exhibiting a context, called the principal context, that distinguishes 
two probabilistic automata whenever there exists a distinguishing context. This leads to the 
notion of a principal trace distribution, which is a trace distribution of a probabilistic automaton 
in parallel with the principal context; the trace distribution precongruence can be characterized 
alternatively as inclusion of principal trace distributions. 

Several other characterizations of the trace distribution precongruence could be found, pos- 
sibly leading to different observational semantics equivalent to the principal trace distribution 
semantics. Further experience with each one of the alternative semantics will determine which 
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Figure 7-3: Trace distribution equivalent probabilistic automata. 

one is more useful. One of the problems with the principal trace distribution characterization 
is that, although from Theorem 7.1.1 there exists an operator || defined on principal traces, 
the definition of || is not simple. For ordinary automata the traces of a parallel composition 
of two automata are exactly those sequences of actions that restricted to each component give 
a trace of the component. This property does not hold for principal trace distributions (cf. 
Example 7.4.1). It is desirable to find a semantics that characterizes the trace distribution 
precongruence and for which the corresponding parallel composition operator has a simple 
definition; however, it is not clear whether such a semantics exists. 

7.2 Trace Distributions 

Let if be a probabilistic execution fragment of a probabilistic automaton M, and let / be a 
function from 0# to = ext(H)*U ext{H) w that assigns to each execution of 0# its trace. The 
trace distribution of H, denoted by tdistr(H), is the probability space completion((Q,,J-,P)) 
where T is the u-field generated by the cones Cp, where (3 is a finite trace of H , and P = /(Pff). 
Observe that, from Proposition 3.1.4, / is a measurable function from (0//, Tu) to (0, J-), since 
the inverse image of a cone is a union of cones. Denote a generic trace distribution by V. A trace 
distribution of a probabilistic automaton M is the trace distribution of one of the probabilistic 
executions of M. Denote by tdistrs(M) the set of the trace distributions of a probabilistic 
automaton M. 

It is easy to see that trace distributions extend the traces of ordinary automata: the trace 
distribution of a linear probabilistic execution fragment a is a distribution that assigns proba- 
bility 1 to trace(a). 

Given two probabilistic execution fragments Hi and H2, it is possible to check whether 
tdistr(Hi) = tdistr(H 2 ) just by verifying that PuistriH^C p] = Ptdistr(H 2 )[Cp] for each finite 
sequence of actions (3. This is an easy consequence of the extension theorem (cf. Theorem 3.1.2). 

Example 7.2.1 (Reason for the definition of J2) The reader may wonder why we have 
not defined to be £race(0#). This is to avoid to distinguish two trace distribution just be- 
cause they have different sample spaces. Figure 7-3 illustrates the idea. The two probabilistic 
automata of Figure 7-3 have the same trace distributions; however, the left probabilistic au- 
tomaton has a probabilistic execution where the trace a°° occurs with probability 0, while the 
right probabilistic automaton does not. Thus, by defining the sample space of tdistr(H) to be 
trace{Q,jj)i the two probabilistic automata of Figure 7-3 would be distinct. In Chapter 8 we 
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define several simulation relations for probabilistic automata, and we show that they are sound 
for the trace distribution precongruence; such a result would not be true with the alternative 
definition of a trace distribution. ■ 

Prefixes 

The notion of a prefix for traces can be extended to the probabilistic framework by following 
the same idea as for the notion of a prefix defined on probabilistic executions (cf. Section 4.2.6). 
A trace distribution V is a prefix of a trace distribution V , denoted by V < V , iff for each 
finite trace (3, Pv[Cfj] < Pv[Cfj]. Thus, two trace distributions are equal iff each one is a prefix 
of the other. 

Lemma 7.2.1 Let Hi and H 2 be two probabilistic execution fragments of a probabilistic au- 
tomaton M. If Hi < H2, then tdistr(Hi) < tdistr(H 2 ). ■ 

Action Restriction 

Similarly to the ordinary case, it is possible to define an action restriction operator on trace 
distributions. Let V = (S7,.F, P) be a trace distribution, and let V be a set of actions. Then 
the restriction of V to V, denoted by V \ V, is the probability space completion((Q,',J-',P')) 
where 0' = \ V , T' is the u-field generated by the sets of cones of 0', and P' is the inverse 
image of P under the function that restricts traces to V. 

Lemma 7.2.2 Let V be a trace distribution. Then (V \ Vi) \ V 2 = V \ (Vi l~l V 2 ). 

Proof. This is a direct consequence of the fact that restricting a trace to V\ and then to V 2 is 
equivalent to restricting the same trace to V\ fl V 2 . Formally, • \ (Vi P\ V 2 ) = (• \ V 2 ) o (• \ V\). ■ 

Finally, we want to show that, if M = Mi\\M 2 , then the projection of a trace distribution of 
M onto Mi and M 2 is a trace distribution of Mi and M 2 , respectively. Formally, 

Proposition 7.2.3 If V G tdistrs{M 1 \\M 2 ), then V \ acts(M{) G tdistrs(Mi), i = 1,2. 

The converse of Proposition 7.2.3 is not true; an illustrating example is given in Section 7.4 
(cf. Example 7.4.1). The rest of this section is dedicated to the proof of Proposition 7.2.3. We 
start with a definition of an internal trace distribution, which is a trace distribution that does 
not abstract from internal actions. 

Let a be an execution of a probabilistic automaton M. The internal trace of a, denoted 
by itrace(a), is the subsequence of a consisting of the actions of M. Let if be a probabilistic 
execution fragment of M, and let / be a function from 0# to = acts(H)* U acts(H) w that 
assigns to each execution of 0# its internal trace. The internal trace distribution of if, denoted 
by itdistr(H), is the probability space completion((Q,,J-,P)) where T is the u-field generated 
by the cones of 0, and P = /(Pff). Observe that, from Proposition 3.1.4, / is a measurable 
function from (£Ih,3~h) to (0, J 7 ). Denote a generic internal trace distribution by V. Denote 
the set of internal trace distributions of a probabilistic automaton M by itdistrs(M). 

Lemma 7.2.4 Let H be a probabilistic execution fragment of a probabilistic automaton M . 
Then, tdistr(H) = itdistr(H) \ ext(H). 
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Proof. This is a direct consequence of the fact that the set of executions of H whose trace 
contains a given (3 is the set of executions of H whose internal trace restricted to the external 
actions of H contains (3. Formally, trace(-) = itrace(-) o (• \ ext(H)). ■ 

Lemma 7.2.5 Let H be a probabilistic execution fragment of M\\\M2, where M\ and Mi are 
two compatible probabilistic automata. Then itdistr(H\Mi) = itdistr(H) \ acts(Mi), i = 1,2. 

Proof. Let V denote itdistr(H\Mi), and let V' denote itdistr(H) \ acts(Mi). We need to 
show that for each finite internal trace (3, PfC/j] = P'\Cp\. Let V" denote itdistr(H). From the 
definition of an internal trace, 

P[Cp] = P H \ Ml [a e tt H \ Ml I P < itrace(a)}. (7.1) 

From the definition of V' and V", 

P'[C P ] = P"\p' e ft" | (1 < (1' \ acts(Mi)]. (7.2) 

From the definition of itdistr(H) and (7.2), 

P'[Cp] = P H [a ett H \fi < itrace(a) \ acts(M t )]. (7.3) 

Thus, from (7.1) and (7.3), we need to show that 

PH\M t [ a ^ ftflTM, I P ^ itrace(a)] = Pff[a G Qh \ ft < itrace(a) \ acts(Mi)]. (7-4) 

By using a characterization of the involved events as a disjoint union of cones, and by rewriting 
Equation 7.4 accordingly, we obtain 

Ph\m\ U C ^ ( 7 - 5 ) 

q£ states (H\Mi)\itrace(q)=l3,Iact(q) = Iact(j3) 

= p h [ U <?*]• 

q(zstates(H )\itrace(q) \ acts(Mi)=f3,lact(q) = lact(f3) 

Observe that for each q £ states(H) such that itrace(q) \ acts(Mi) = (3 and lact(q) = 
lact(P), the state q\Mi is a state of H\Mi such that itrace(q\Mi) = (3 and lact(q\Mi) = 
lact(P). Moreover, the states q of the left expression of (7.5) are partitioned by the relation 
that relates q and q' whenever q\Mi = q'\Mi. Thus, if we show that for each trace (3 and each 
q G states(H\Mi) such that itrace(q) = (3 and lact(q) = lact((3), 

PH\M t [Cq] = P H[ l Jq'eq]H\lact(q') = lact(l3)Cq']^ ( 7 - 6 ) 

Equation (7.5) is proved. Observe that 

Pn[ l Jq'estates(H)\q'\M,=q,lact(q') = lact(l3)Cq'] = 2^ Ph[C q'}, (7.7) 

q'(zmin(q~\H) 

since {q' £ states(H) \ q'\Mi = q,lact(q r ) = lact((3)} = min(q\H). Thus, Equation (7.6) 
becomes 

PH\ Mi [C q ]= E p H[C q >], (7.8) 

q'(zmin(q~\H) 

which is true from Proposition 4.3.5. ■ 
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Lemma 7.2.6 Let H be a probabilistic execution fragment of M\\\M 2 , where Mi and M 2 are 
two compatible probabilistic automata. Then tdistr(H\Mi) = tdistr(H) \ acts(Mi). 

Proof. From Lemma 7.2.4, 

tdistr ( H \Mi ) = itdistr (H\Mi) \ ext(Mi). (7.9) 

From Lemma 7.2.5 and (7.9), 

tdistr(H\Mi) = (itdistr (H) \ acts(Mi)) \ ext(Mi). (7.10) 

From Lemma 7.2.2 and (7.10), 

tdistr(H\Mi) = (itdistr (H) \ ext(Hj) \ acts(Mi). (7.11) 

From Lemma 7.2.4 and (7.11), 

tdistr(H\Mi) = tdistr (H) \ acts(Mi), (7.12) 

which is what we needed to prove. ■ 

Proof of Proposition 7.2.3. Let V £ tdistrs(Mi\\M 2 ) ■ Then there exists a probabilis- 
tic execution H of Mi||M2 such that tdistr(H) = V. From Proposition 4.3.4, H\Mi is a 
probabilistic execution of M 8 -. From Lemma 7.2.6, tdistr (H\Mi) = V \ acts(Mi). Thus, 
V \ acts (M{) £ tdistr s(Mi). ■ 

7.3 Trace Distribution Preorder 

Once trace distributions are defined, the trace distribution preorder can be defined as trace 
distribution inclusion. Formally, let M\,Mi be two probabilistic automata with the same 
external action signature. The trace distribution preorder is defined as follows. 

Mi C D M 2 iff tdistrs(M 1 ) C tdistrs(M 2 ). (7.13) 

The trace distribution preorder is a conservative extension of the trace preorder of ordinary 
automata, and it preserves properties that resemble the safety properties of ordinary automata 
[AS85]. Here we give some examples of such properties. 

Example 7.3.1 The following property is preserved by the trace distribution preorder. 

" After some finite trace (3 has occurred, then the probability that some other trace 
fi' occurs, is not greater than p." 

In fact, suppose that M\ C^ M 2 , and suppose that M 2 satisfies the property above, while 
Mi does not. Then there is a trace distribution of Mi where the probability of fi' after (3 
conditional to (3 is greater than p. Since Mi C^ M 2 , there is a trace distribution of M 2 where 
the probability of fi' after (3 conditional to (3 is greater than p. This contradicts the hypothesis 
that M 2 satisfies the property above. Observe that the property above would still be preserved 
if we replace fi' with a set of traces. ■ 
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Example 7.3.2 The following property is preserved by the trace distribution preorder. 

LL In every computation where infinite external activity occurs with probability 1, if 
a finite trace (3 occurs, then the probability that some other trace (3' occurs after (3 
given that (3 occurs is at least p." 

A more concrete instantiation of the property above is "under the hypothesis that a distributed 
system never deadlocks, every request of service eventually gets a response with probability at 
least p". This property is definitely more interesting than the property of Example 7.3.1 since it 
involves a progress statement, one of the property of key interest for the analysis of randomized 
distributed algorithms. Thus, if in a system it is always possible to avoid a deadlock, under 
the assumption that we always schedule a transition and under the condition that no infinite 
internal computation is possible, the property above guarantees progress. However, in order to 
be sure that if Mi C^ Mi and Mi satisfies the property above then M\ guarantee progress, we 
need to make sure that from every state of Mi it is possible to avoid deadlock and there is no 
possibility of infinite internal computation. Such a property must be verified separately since it 
is not guaranteed by the trace distribution preorder. Fortunately, there are several cases (e.g., 
n processes running in parallel that communicate via shared memory) where it is easy to verify 
that it is always possible to avoid a deadlock. 

To prove that the property above is preserved, suppose that Mi C^ M2, and suppose that 
M2 satisfies the the property above, while Mi does not. Then there is a trace distribution of Mi 
with infinite external computation where the probability of (3' after (3 conditional to (3 is greater 
than p. Since Mi C^ M2, there is a trace distribution of M2 with infinite external computation 
where the probability of (3' after (3 conditional to (3 is greater than p. This contradicts the 
hypothesis that M2 satisfies the property above. ■ 

Example 7.3.3 The following property is preserved by the trace distribution preorder. 

LL In every computation where infinite external activity occurs with probability 1, if a 
finite trace (3 occurs, then, no matter what state is reached, a trace (3' occurs ofter 
(3 with probability at least p." 

A more concrete instantiation of the property above is "under the hypothesis that a distributed 
system never deadlocks, if a process has requested a service (/3), then, no matter what state is 
reached, either the service has received a positive acknowledgment already (/?'), or a positive 
acknowledgment will be received eventually with probability at least p". This property is pre- 
served by the trace distribution preorder since it is equivalent to the property of Example 7.3.2 
with p = 1 (cf. Proposition 5.5.5 to have an idea of why this is true). ■ 

Essentially, the rule of thumb to determine what properties can be guaranteed to be preserved 
under the trace distribution preorder is the following: express the property of interest as a 
property (f> of the trace distributions of a probabilistic automaton M plus a condition ip on the 
structure of M. If Mi C^ M2, then the trace distributions of Mi satisfy the property <p. Thus, 
if we know that M2 satisfies the property of interest, it is enough to verify separately that Mi 
satisfies ip in order to be guaranteed that also Mi satisfies the property of interest. 
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Figure 7-4: The trace distribution preorder is not a precongruence. 
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Figure 7-5: A probabilistic execution of M2IIC. 

7.4 Trace Distribution Precongruence 

Although the trace distribution preorder preserves some properties that are useful for the anal- 
ysis of randomized distributed systems, the trace distribution preorder is not a precongruence, 
and thus it does not allow us to use modular analysis. 

Example 7.4.1 (The trace distribution preorder is not substitutive) Consider the two 
probabilistic automata Mi and Mi of Figure 7-4. It is easy to check that M\ and Mi have 
the same trace distributions. Consider now the context C of Figure 7-4. Figure 7-5 shows a 
probabilistic execution of M2HC where there is a total correlation between the occurrence of 
actions d and / and actions e and g. Such a correlation cannot be obtained from Mi||C, since 
the choice between / and g must be resolved before knowing what action among d and e is 
chosen probabilistically. Thus, Mi||C and M2HC do not have the same trace distributions. ■ 

This leads us to the definition of the trace distribution precongruence, denoted by Qdc, as the 
coarsest precongruence that is contained in the trace distribution preorder. This definition of the 
trace distribution precongruence is not constructive, and thus it is difficult to understand what 
we have defined. Furthermore, we do not have any observational semantics that characterizes 
the trace distribution precongruence. In Section 7.5 we give an alternative characterization 
of the trace distribution precongruence that gives a better idea of the relation that we have 
defined. Here we give some examples of properties that are preserved by the trace distribution 
precongruence and that are not preserved by the trace distribution preorder. 

Example 7.4.2 The following property is preserved by the trace distribution precongruence 
but not by the trace distribution preorder. 
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" After some finite trace (3 has occurred, no matter what state is reached, the prob- 
ability that some other trace fi' occurs from the state reached is not greater than 
p." 

This property is not preserved by the trace distribution preorder since trace distributions cannot 
detect all the points where we may start to study the probability of (3' to occur. However, this 
task is possible with the help of an external context. We use a context C that performs a fresh 
action o and then stops. 

Suppose that Mi C_dc M2 and suppose that Mi satisfies the property above, while M\ 
does not. Then there is a probabilistic execution H.\ of M\ where some state q is reached after 
the occurrence of /3, and the probability that fi' occurs from q is greater than p. Consider a 
probabilistic execution H[ of Mi||C such that H[ \M\ = H\ and such that action o is scheduled 
exactly from the minimal state q' such that q'\M\ = q. Then, o occurs always after /3, and 
the conditional probability of fi' after o given that o occurred is greater than p in the trace 
distribution of H[. Since Mi C_dc M2, then there is a probabilistic execution H 2 of M2HC 
whose trace distribution is the same as the trace distribution of H 2 . This means that there is at 
least one state q" in H 2 , reached immediately after the occurrence of o, where the probability 
that fi' occurs from q" in H 2 is greater than p. Consider H 2 \M 2 , and change its transition 
relation to obtain a probabilistic execution H 2 such that H 2 \>(q"\M 2 ) = (H 2 \M 2 )\>(q"\M 2 ). 
Then the probability that fi' occurs from q"\M 2 in H 2 is greater than p. Moreover, (3 has 
occurred when q\M 2 is reached. This contradicts the hypothesis that M 2 satisfies the property 
above. ■ 

Example 7.4.3 The following property is preserved by the trace distribution precongruence 
but not by the trace distribution preorder. 

LL In every computation where infinite external activity occurs with probability 1, if a 
finite trace (3 occurs, then, no matter what state is reached, if another trace fi" has 
not occurred yet after (3, then a trace (3' occurs with probability at least p." 

A more concrete instantiation of the property above is "under the hypothesis that a distributed 
system never deadlocks, if a process has requested a service (/3) and has not received yet a 
refusal (/?") then, no matter what state is reached, a positive acknowledgment (/?') will be 
received eventually with probability at least p". Observe that the main difference from the 
property of Example 7.3.3 is in the use of fi" . The presence of fi" does not guarantee that fi' 
occurs with probability 1. 

Even in this case in the proof we use a context C with a fresh action o. Suppose that 
Mi C_dc M2 and suppose that M2 satisfies the property above, while Mi does not. Then there 
is a probabilistic execution Hi of Mi where infinite external activity occurs such that there is a 
state q of Hi that is reached after the occurrence of (3 and before the occurrence of /3", and such 
that the probability that fi' occurs from q is smaller than p. Consider a probabilistic execution 
H'i of Mi||C such that H[\Mi = Hi and such that action o is scheduled exactly from the 
minimal state q' such that q'\Mi = q. Then, o occurs always after (3 and before fi" occurs after 
/3, and the conditional probability of fi' after o given that o occurred is greater than p in the 
trace distribution of H[. Since Mi C_dc M2, then there is a probabilistic execution H 2 of M2HC 
whose trace distribution is the same as the trace distribution of H 2 . This means that there is at 
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Figure 7-6: The principal context (left) and the simple principal context (right). 

least one state q" in H 2 , reached immediately after the occurrence of o, where the probability 
that fi' occurs from q" in H 2 is smaller than p. Consider H' 2 \M2, and change its transition 
relation to obtain a probabilistic execution Hi such that H2>(q"\M2) = (H' 2 \M^ )>(<?" [M2). 
Then the probability that fi' occurs from q"\M2 in H2 is smaller than p. Moreover, (3 has 
occurred when q\M2 is reached and similarly (3" has not occurred after the occurrence of (3. 
This contradicts the hypothesis that M2 satisfies the property above. ■ 

7.5 Alternative Characterizations of the Trace Distribution 
Precongruence 

In this section we give an alternative characterization of the trace distribution precongruence 
that is easier to manipulate. We define a principal context, denoted by Cp, and we show that 
there exists a context C that can distinguish two probabilistic automata Mi and M2 iff the 
principal context distinguishes Mi and M 2 . 

7.5.1 The Principal Context 

The principal context is a probabilistic automaton with a unique state and three self-loop tran- 
sitions labeled with actions that do not appear in any other probabilistic automaton. Two 
self- loop transitions are deterministic (Dirac) and are labeled with action left and right, respec- 
tively; the third self-loop transition is probabilistic, where one edge leads to the occurrence of 
action pleft with probability 1/2 and the other edge leads to the occurrence of action pright 
with probability 1/2. Figure 7-6 shows the principal context. 

The principal context is not a simple probabilistic automaton; however, since it does not 
have any action in common with any other probabilistic automaton, the parallel composition 
operator can be extended trivially: no synchronization is allowed. Alternatively, if we do not 
want a non-simple context, we can replace the principal context with the simple principal 
context, represented in Figure 7-6, as well. In this case we need to assume that also action start 
does not appear in any other probabilistic automaton. The main theorem is the following. 

Theorem 7.5.1 M x Q DC M 2 iff M^C'p Q d M 2 \\C p . ■ 

As a corollary we obtain an alternative characterization of the trace distribution precongruence 
and a compositional observational semantics for probabilistic automata. A principal trace distri- 
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button of a probabilistic automaton Mis a trace distribution of M||Cp. Denote by ptdistrs(M) 
the set tdistrs{M\\Cp). 

Corollary 7.5.2 M x Edc M 2 iff ptdistrsiM-^) C ptdistrs{M 2 ). ■ 

The fact that the principal context is not a simple probabilistic automaton may appear to 
be confusing. Here we shed some light on the problem. First of all, in Chapter 4 we have 
defined parallel composition only for simple probabilistic automata; in this section, in order to 
account for the principal context, we have extended parallel composition to pairs of probabilistic 
automata, not necessarily simple, that do not have any action in common. This raises an 
immediate question: is the trace distribution precongruence defined based solely on contexts 
that are simple probabilistic automata or is it defined based on any compatible context according 
to the new extended parallel composition? The answer to this question, as it will become clear 
from the proof of Theorem 7.5.1, is that it does not matter because the two definitions are 
equivalent. That is, if there is a non-simple context that distinguishes two simple probabilistic 
automata Mi and M 2 , then the simple principal context distinguishes Mi and M 2 as well. 

Our choice of the principal context is just stylistic since it contains less structure than 
the simple principal context. The reader should keep in mind that there are infinitely many 
contexts with the same properties as the principal and the simple principal contexts; any one 
of those contexts can be chosen to give an alternative characterization to the trace distribution 
precongruence. 

7.5.2 High Level Proof 

The rest of this section is dedicated to the proof of Theorem 7.5.1. The proof is structured 
in several steps where at each step a generic distinguishing context C is transformed into 
a simpler distinguishing context C" . The proof of each transformation step is structured as 
follows. Given a distinguishing context C for Mi C^ M 2 , build a simpler context C". Suppose 
by contradiction that C" is not a distinguishing context and consider a trace distribution V of 
Mi||C that is not a trace distribution of M2HC. Let H\ be a probabilistic execution of Mi||C 
such that tdistr(Hi) = V. Transform H\ into a probabilistic execution H[ of Mi||C", and show 
that if there is a probabilistic execution H' 2 of M2IIC" such that tdistr(H' 2 ) = tdistr(H[), then 
H' 2 can be transformed into a probabilistic execution H 2 of M2HC such that tdistr(H 2 ) = V. 
This leads to a contradiction. 

The high level proof of Theorem 7.5.1 is then the following. 

=^: Assuming that the principal context distinguishes Mi and M2, we show that the simple 
principal context distinguishes Mi and M2. 

^=: We consider a generic context C that distinguishes Mi and M2, and we transform it into 
the principal context, showing that the principal context distinguishes Mi and M2. The 
transformation steps are the following. 

1. Ensure that C does not have any action in common with Mi and M2 (Lemma 7.5.3); 

2. Ensure that C does not have any cycles in its transition relation (Lemma 7.5.4); 

3. Ensure that the branching structure of C is at most countable (Lemma 7.5.5); 
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4. Ensure that the branching structure of C is at most binary (Lemma 7.5.6); 

5. Ensure that the probabilistic transitions of C lead to binary and uniform distributions 
(Lemma 7.5.7); 

6. Ensure that each action of C is external and appears exactly in one edge of the 
transition relation of C (Lemma 7.5.8); 

7. Ensure that each state of C enables two deterministic transitions and one probabilis- 
tic transition with a uniform binary distribution (Lemma 7.5.9); 

8. Rename all the actions of the context of 7 according to the action names of the 
principal context and then collapse all the states of the new context into a unique 
state, leading to the principal context (Lemma 7.5.10). 

7.5.3 Detailed Proof 

Lemma 7.5.3 Let C be a distinguishing context for two probabilistic automata M\ and Mi- 
Then there exists a distinguishing context C" for Mi and Mi with no actions in common with 
Mi and Mi- C" is called a separated context. 

Proof. The context C" is built from C be replacing each action a in common with Mi and M2, 
called a shared action, with two new actions 0,1,0,2, and by replacing each transition (c, a, V) of 
C with two transitions (c,ai,c r ) and (c 1 ,a2,V), where c' denotes a new state that is used only 
for the transition (c,a,V). We denote c' also by ct c ^ a j>\ when convenient. We also denote the 
set of actions of the kind ai and 02 by V\ and V2, respectively. 

Let V be a trace distribution of Mi||C that is not a trace distribution of M2IIC. Consider a 
probabilistic execution Hi of Mi||C such that tdistr(Hi) = V, and consider the scheduler that 
leads to H\. Apply to Mi||C" the same scheduler with the following modification: whenever a 
transition ((«i, c), a,Vi ® V) is scheduled in Mi||C, schedule ((«i, c), ai,V((si, c'))), where d is 
c (c,a,V)i followed by ((«i, c'), a, Vi®V(c')), and, for each s^ 6 fii, followed by ((s^, c'), a2,T>(s' 1 )(x) 
V). Denote the resulting probabilistic execution by H[ and the resulting trace distribution by 
V. Then, 

V \ acts(Mi\\C) = V. (7.14) 

To prove (7.14) we define a new construction, called collapse and abbreviated with dp, to be 
applied to probabilistic executions of M 8 ||C", i = 1,2, where each occurrence of a shared action 
a is followed immediately by an occurrence of its corresponding action 02- 

Let H' be a probabilistic execution of M 8 ||C" where each occurrence of a shared action a is 
followed immediately by an occurrence of its corresponding action 02- For convenience denote 
clp(H') by if. A state q of H' is closed if each occurrence of a shared action a is followed 
eventually by an occurrence of the corresponding action 02- For each closed state q of H' , let 
clp(q) be obtained from q as follows: each sequence 

(s , c )ai(s , c ir )r 2 (s 2 , c tr ) ■ ■ ■ n(s k , c tr )a(s, c tr )a 2 (s, c) 

is replaced with 

(•So,c )r 2 (s2,co) • • ■T k (s k ,c )a(s,c), 
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and each sequence 

(s ,co)ai(si,c ir )r 2 (s2,c ir ) • • -r fc (s fc ,c ir ) 
occurring at the end of q is replaced with 

(•So,co)r 2 (s2,co) • • ■T k (s k ,c ). 
Define 

states(H) = {clp(q) \ q £ states(H'), closed(q)} . (7-15) 

Let (q, V) be a restricted transition of H' where q is a closed state, and suppose that no action 
of V\ U V2 occurs. Consider a pair (a, q') of S7. If a is not a shared action, then let 

V {a<ql) = V({a,clp(q')))- (7.16) 

if a is a shared action, then let 

%, q >) = {(a,clp(q"))\(a 2 ,q")en$'}, (7.17) 

and for each (a, g'") £ i7( ag /), let 

P(a, q >)[(a,^")] = PA^Xclp-^q'")}, (7.18) 

where for each state q of H , c/p~ (g) is the set of closed states q' of if' such that clp(q') = q. 
The transition clp((q,V)) is defined to be 

clp((q,P)) = lclp(q), J2 P[(a,q')]V^ ql) \ . (7.19) 

V (a,q')en J 

For the transition relation of H , consider a state q of H Let min(clp~ (qj) be the set of minimal 
states of c/p _1 (g) under prefix ordering. For each state q £ c/p _1 (g), let 

p£'? _1 (9) ^ P g'[ C d ( 7 _20) 

The transition enabled in H from q is 

X) pf~ 1{q) Pf [acts{Mi\\C)\clp{trf \ acts(M,\\Cj). (7.21) 

g'Gc/p-^g) 

Note the similarity with the definition of the projection of a probabilistic execution fragment 

(cf. Section 4.3.2). 

The probabilistic execution H satisfies the following properties. 

a. if is a probabilistic execution of M 8 ||C. 

The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of if is a finite execution fragment of M 8 ||C follows from a simple 
analysis of the definition of dp. 

From (7.21) it is enough to check that for each closed state q' of if', the transition 
clp(tr^, \ acts(Mi\\C)) is generated by a combination of transitions of M 8 ||C. Since tr^, 

is a transition of R\ (^ r v \ acts(Mi\\C)) can be expressed as J2jPj(q' ~ trj), where each 
tr-j is a transition of M 8 ||C". We distinguish three cases. 
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1. tr-j is a non-shared transition of M 8 -. 



Then tr-j = ((s,c),a,V ® 7?(c)) for some action a and probability space V, where 
(s,c) = Istate(q'). Let lstate(dp(q')) = (s',c'). Then, s' = s, as it follows directly 
from the definition of dp. Define tr'- to be the transition ((s, c'), a,V® V(c')). Then 
fr'- is a transition of M 8 ||C and dp(q' ' ~ frj) = dp(q') ~ tr'- 

2. £rj is a non-shared transition of C". 

Then £rj = ((s, c), a,V(s) ® 7-*) for some action a and probability space V, where 
(s,c) = Istate(q'). Let lstate(dp(q')) = (s',c'). Then, s' = s and c' = c, as it follows 
directly from the definition of c/p after observing that g' must be a closed state in 
order to enable tr-j. Define tr'- to be tr-j. Then fr'- is a transition of M 8 ||C and 
dp(q' ~ frj) = dp(q') ~ tr'- 

3. £rj is a shared transition. 

Then tr-j = ((s, c ir ), a,V ® V(c ir )) for some action a and probability space 7-*, where 
(s,c ir ) = Istate(q'). In particular, c ir is one of the states that are added to those 
of C, and tr is a simple transition of C with action a. Moreover, from each state 
(V,c ir ) G £l-pQT>{c t r)i there is a transition ((V, c ir ), a,2,V(s') ® Vf r ) enabled. Let 
lstate(dp(q')) = (s',c'). Then, s' = s. Define fr'- to be ((s,c'),a,V ®Vf r )- Then, 
from the definition of C", fr'- is a transition of M 8 ||C. 

Observe that dp distributes over combination of transitions. Moreover, from Equa- 
tion (7.19), observe that for each j dp(q' ~ trj) = dp(q') ~ tr'-. Thus, dp(tr^, \ 
acts(Mi\\C)) = dp(q')~ (J2j Pjtr'j), which is generated by a combination of transitions of 

Mi\\C. 

b. For each state q of H , 

P H [C q }= Y, Pw[C q <\. (7.22) 

q' (zTtiin(clp~ (g)) 

This is shown by induction on the length of q. If q consists of a start state only, then the re- 
sult is trivial. Otherwise, from the definition of the probability of a cone, Equation (7.21), 
and a simple algebraic simplification, 

P H [C qas ] = P H [C q ] I ^ p c 'r 1{q) F q ,(qas)) , (7.23) 




where F q t(qas) expresses the probability of the completions of q' to a state whose col- 
lapse gives qas without using actions from V\ U V 2 in the first transition. Formally, 
if a is not a shared action, then F q i(qas) is P*J [a X dp~ (qas)]; otherwise, F q i(qas) 
is Pfi'[(a,q'a(s' ,c tr ))]P^ s ,^[(a 2 ,q'a(s' ,c tr )a 2 (s' ,c))], where c tr = lstate(q')\C", and 
s = (s',c). In the first case, fl^, n ({a} X dp~ 1 (qasj) contains only one element, say 
(a,q'as"), and PH'[C q i]F q t(qas) gives PH'[C q ' as "]] in the second case PH'[C q i]F q t(qas) 

gives Pff'[C(g'a( S ',c tr )a 2S )]- 
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Observe that the states of min(clp~ (qasj) are the states of the form described above 
(simple cases analysis). Thus, by applying induction to (7.23), using (7.20), simplifying 
algebraically, and using the observations above, 

P H [C qas }= Y, Pw[C q ,]. (7.24) 

q' £min(clp~ (qas)) 

c. tdistr(H) = tdistr(H') \ acts(M t \\C). 

Let (3 be a finite trace of H or H' . Then {a £ Qffi \ (3 < trace(a) \ acts(Mi\\C)} can be 
expressed as a union of disjoint cones U ge eCg where, if the last action of (3 is a and a is 
not a shared action, 

= {q e states(H') \ trace(q) \ acts(M t \\C) = (3, lact(q) = a}, (7.25) 

and if the last action of (3 is a and a is a shared action, 

= {q e states(H') \ trace(q) \ acts(M t \\C) = (3, lact(q) = a 2 }. (7.26) 

Observe that is a set of closed states. The set clp(Q) is the set 

clp(Q) = {q G states(H) \ trace(q) = (3, lact(q) = a}, (7.27) 

which is a characterization of {a £ S7# | (3 < trace(a)} as a union of disjoint cones. 
Observe that min(clp~ (clp(Q)j) = 0. Moreover, for each q\ ^ q 2 of c/p(0), clp~ (q\) n 
c/p _1 (g2) = 0- Thus, from (7.22), P H ,[\J qe& C q ] = PH[U q eclp(&)C g ]. This is enough to 
conclude. 

To complete the proof of (7.14) it is enough to observe that Hi = clp(H[). Property (7.14) is 
then expressed by property (c). 

Suppose by contradiction that it is possible to obtain V from M2IIC". Consider the scheduler 
that leads to V in M2IIC", and let H' 2 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution H" of M2IIC" whose trace distribution is V , and such that 
each shared action a is followed immediately by its corresponding action a 2 . Then we let H 2 be 
clp^HlJ). This leads to a contradiction since tdistr(H 2 ) = V. The rest of the proof is dedicated 
to the construction of H 2 . 

For each state q of H' 2 , let exch(q) be the set of sequences q' that can be obtained from q 
as follows: each sequence 

(s ,c ir )a(si,c ir )r 2 (s2,c ir ) • • ■T h {s h ,Ci r )a 2 {sh,c) 
is replaced with 

(s , c ir )a(s x , c ir )a 2 (si, c)r 2 (s 2 , c) • • -T h {s h , c), 
each sequence 

(s ,c ir )a(si,c ir )r 2 (s 2 ,c ir ) • • -T h (s h , c tr ) 
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occurring at the end of q is replaced with 

(s , c tr )a(s 1 , c tr )a 2 (s 1 , c)t 2 (s 2 , c) • • -T h {s h , c), 
where c is any of the states that a 2 may lead to from c ir , and each sequence 

(s ,c ir )a(si,c ir ) 
occurring at the end of q, where a is a shared action, either it is replaced with 

(s ,c ir )a(s 1 ,c ir )a 2 (s 1 ,c), 
where c is any of the states that a 2 may lead to from c ir , or it is not replaced. Then, define 

states(Hlj) = \J exch(q). (7.28) 

q(zstates(Hl) 

Let (q,V) be a restricted transition of H' 2 , and suppose that no action of V 2 occurs. Let q' be 
a state of exch(q) that does not end with a shared action. Then, for each (a,qi) G there is 
exactly one q[ G exch(qi) such that q' < q[ and \q[\ = \q'\ + 1 (simple analysis of the definition 
of exch). Denote such q[ by exch q t(q\). Let 0' = {(a, exch q t(q\) \ (a,qi) G 0}, and let, for each 
(a, g x ) G 0', -P'[(«, ^i)] = -P[(« X ea;c/i _1 (g 1 ))], where exch~ x {q) is the set of states q' of i/^ such 
that q G exch(q'). Then define the transition exch q t((q,V)) to be 

exch q ,((q,V)) = (q',V). (7.29) 

For each state g of if", let min(exch~ (qj) be the set of minimal states of exch~ (q) under 
prefix ordering. For each state q' of e;rc/j _1 (g), where q is closed, let 

• p q , = P H i[C q i] if q' is closed, i.e., if each occurrence of a shared action a is followed 
eventually by an occurrence of its corresponding action a 2 ; 

• p q , = Pfji[C q i]Pt r [c] if g' is open, where lstate(q')\C" = c ir and lstate(q)\C = c. 
For each q' G exch~ x {q), let 



P g T A " 1(9) ^ = ^ -. (7.30) 

2—iq"£min(exch 1 (q)) i q" 

If the last action of q is a shared action a, and Istate(q) = (s,c ir ), then the transition enabled 
from q in if" is 

<T ((s,c ir ),a 2 ,D(s)®P ir ). (7.31) 

If the last action of q is not a shared action, then the transition enabled from q in if" is 

J2 p;r h ~ 1{9) rfHacts(Hti\V 2 ]exch q (tr** \ (acts(H' 2 )\V 2 )). (7.32) 

q'(zexch~ (g) 

The probabilistic execution H' 2 satisfies the following properties. 
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a. H" is a probabilistic execution of M2IIC". 

The fact that each state of H" is reachable can be shown by a simple inductive argument; 
the fact that each state of H" is a finite execution fragment of M2IIC" follows from a 
simple analysis of the definition of exch. 

We need to check that for each state q of H" the transition enabled from q in H" is 
generated by a combination of transitions of M2IIC". If the last action of q is a shared 
action, then the result follows immediately from Expression (7.31) and the definition of 
C . If the last action of q is not a shared action, then consider a state q' £ exch~ (q). 

TTl 

The transition tr , 2 \ (acts(H' 2 )\V2) can be expressed as Y^iViW ~ ^ r «)? where each tr-{ is 
a transition of M2IIC" enabled from Istate(q'). We distinguish three cases. 

1. tri is a non-shared transition of M 2 . 

Then tr-{ = ((s,c),a,V ® D(c)) for some action a and probability space V, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s' = s. Define tr\ to be the 
transition ((s, c'), a,V®V(c')). Then fr'- is a transition of M2IIC" and exch q (q'^tri) = 
q~ tr\. 

2. £r 8 - is a non-shared transition of C". 

Then £r 8 - = ((s, c), a,X>(s) ® 7-*) for some action a and probability space 7-*, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s' = s and c = c'. Define fr'- to be 
fr,-. Then fr'- is a transition of M2IIC" and exch q (q' " tr{) = q ~ tr\. 

3. £r 8 - is a shared transition. 

Then tr-{ = ((s,c),a,V ® 7?(c)) for some action a and probability space T 7 , where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s' = s and c = c'. Define fr'- to be 
fr,-. Then fr'- is a transition of M2IIC" and exch q (q' ' ~ fr,-) = q^ tr[. 

Observe that exc/j distributes over combination of transitions. Thus, exch q ((tr q t) \ 
(acts(H' 2 )\V 2 )) can be expressed as ^iP«'(<7~ ^ r ')' which i s generated by a combination of 
transitions of M2IIC". From (7.32), the transition enabled from q in H 2 is generated by a 
combination of transitions of M 2 \\C". 



b. For each state g of if", 

^'[Q] = i 5«' fe ""»^ eft ;^:^-"" J " ? ~T ' (7.33) 



E g ' emm ( ea;c A- 1 (g)) P ^ [Cg'] if 1 ends with a shared action, 
IveminCescA- 1 ^)) Pg' otherwise. 



The proof is by induction on the length of q. If q consists of a start state only, then the 
result is trivial. Otherwise, consider PH"[C qas \- We distinguish two cases. 

1. q is open. 

In this case, since in H 2 each shared action is followed immediately by the corre- 
sponding action of V 2 , a is an action of V 2 . Moreover, from the definition of exch, 

exch~ (q) = min(exch~ (qasj) = min(exch~ (<?)), (7.34) 

and all the elements of exch~ (q) are open states. From induction, 

P H >>[C q ]= £ PH>[C q >]. (7.35) 

q f £min(exch~ (g)) 
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Let c = s\M2, and let c tr = lstate(q)\C . Then, for each q' £ min(exch (<?)), 
c tr = lstate(q')\C, and 

Pi? 3 = P H >[C q >]Ptr[c\. (7.36) 

TTII 

Moreover, P q 2 [(a, qas)] = Pt r [c\. Thus, from the definition of the probability of a 
cone and (7.35), 

P H »[C q as]= E PHl[C q >]Ptr[c\. (7.37) 

q '(zTtiin(exch~ (g)) 

By using the fact that min(exch~ 1 (qj) = ■min(exch~ 1 (qasj), and using (7.36), we 
obtain 

PH»[c qa .]= E p7- ( 7 - 38 ) 

q '(zTtiin(exch~ (gas)) 

2. q is closed. 

In this case, from the definition of the probability of a cone and (7.32), 

P H »[C q as] = P H »[C q ] l £ pT h ~ 1{q) Pf[a X exch-^qas)]] . (7.39) 

Xq'eexch,- 1 ^) / 

Let Ptr q [q'] denote Pi r [c], where c = lstate(q)\C", and c ir = lstate(q')\C" . Then, 
from induction and (7.30), 

P H »[C qa s] = E PH^C^pfiaxexch-^qas)]^ (7.40) 

q f £ exch (q)\closed(q f ) 

E PH'iC^Ptr^pfiiaxexch-^qas)]. 

q '(zexch~ (g)| open( q ') 

We distinguish two subcases. 

(a) a is a shared action. 

In this case each state q' of exch~ x {q) such that Pi 2 [a X exch~ 1 (qas)] > 
is closed. Thus, only the first summand of (7.40) is used. Moreover, each 
state of min(exch~ 1 (qasj) is captured by Expression (7.40). Thus, P H i[C qas ] = 
52 q '€mi n {exch- 1 {qas)) P Hl[Cq>]- Observe that qas is open. 

(b) a is not a shared action. 

In this case, for each q' £ exch~ (g), if q' is closed, then all the states reached in 
£l q i fl ({a} X exch -1 (qas)) are closed, and if q' is open, then all the states reached 
in fl q i fl ({a} X excbT (gas)) are open. Moreover, each state of min(exch~ (qas)) 
is captured by Expression (7.40). Thus, from the definition of p 9 ? s , Pff'[C qas ] = 

T. q < em in( e xch-\ q «s))PT ■ 0bserVe that aC1S 1S cl ° Sed - 

c. tdistr(H' 2 ) = tdistr(H%). 

Let (3 be a finite trace of H' 2 or H". Then {a £ Cljji \ (3 < trace(a)} can be expressed as 
a union of disjoint cones U ge eCg where 

= {q £ states(H') \ trace(q) = /3, lact(q) = lact(fi)}. (7-41) 
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We distinguish two cases. 

1. fi does not end with an action of V 2 . 

The set 0' = {q £ exch(Q) \ lact(q) = lact(fi)} is a characterization of {a £ Oj^// | 
fi < trace(a)} as a union of disjoint cones. Observe that min( exch~ x (©')) = and 
that for each pair of states q\ ^ q 2 of 0', min(exch~ (qij) C\ min(exch~ (^2)) = 
0. Thus, if fi ends with a shared action, then (7.33) is sufficient to conclude that 
P H i[{a £ £l H i I fi < trace(a)}] = P H n[{a £ £l H n \ fi < trace(a)}]; if fi does not 
end with a shared action, then, since all the states of are closed, Equation (7.33) 
together with the definition of p q , are sufficient to conclude. 

2. fi ends with an action of V 2 . 

In this case fi = fi' ' a 2 for some action a 2 £ V2. Observe that, both in H 2 and H 2 , 
after the occurrence of a shared action a the corresponding action a 2 occurs with 
probability 1: for E' 2 recall that tdistr{H' 2 ) \ acts(M 2 \\C) = V; for E' 2 ' see (7.31). 
Thus, the probability of fi is the same as the probability of /3', and the problem is 
reduced to Case 1. ■ 

Lemma 7.5.4 Let C be a distinguishing separated context for two probabilistic automata Mi 
and M 2 . Then there exists a distinguishing cycle-free separated context C" for Mi and M 2 . 

Proof. C" can be built by unfolding C. Every scheduler for M 8 ||C can be transformed into a 
scheduler for M 8 ||C" and vice versa, leading to the same trace distributions. ■ 

Lemma 7.5.5 Let C be a distinguishing cycle-free, separated context for two probabilistic au- 
tomata Mi and M 2 . Then there exists a distinguishing cycle-free separated context C" for Mi 
and M 2 with a transition relation that is at most countably branching. 

Proof. Let V be a trace distribution of Mi||C that is not a trace distribution of M2IIC. 
Consider the corresponding probabilistic execution H . Observe that H has at most countably 
many states, and that at each state of H there are at most countably many transitions of C 
that are scheduled. Thus, in total, only countably many transitions of C are used to generate 
V. Then C" is C without the unused transitions. ■ 

Lemma 7.5.6 Let C be a distinguishing cycle-free, separated context for two probabilistic au- 
tomata Mi and M 2 such that the transition relation of C is at most countably branching. Then 
there exists a distinguishing cycle-free separated context C" for Mi and M 2 that at each state 
either enables two deterministic transitions or a unique probabilistic transition with two possible 
outcomes. C" is called a binary separated context. 

Proof. For each state s of C, choose a new action start s . Let s enable the transitions 
tr\, tr 2 , . . ., where each tr\ is a transition (s, a,-, V{). The transition relation of C" is obtained in 
two phases. First, a transition is chosen nondeterministically as shown in Figure 7-7, where each 
symbol • denotes a distinct state and each symbol r denotes a distinct internal action; then, for 
each state •;, the transition tr\ is encoded as follows. Let 0,- be {si,i,Si,2 5 • • •}, Pi,j = Pi[ s i,j]> 
and pij = J2k>jPi,k- The transition relation from •; is represented in Figure 7-8, where each 
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Figure 7-7: Nondeterministic choice of a transition. 




Figure 7-8: Transforming a transition into binary transitions. 

symbol • denotes a distinct state and each symbol r denotes a distinct internal action. Observe 
that by scheduling all the transitions of the diagram above, for each j we have 



P[Si,j] = Pi[Si: 



31i 



f7.42) 



where P[sij] is the probability of reaching s 8J - from •;. Denote the set of actions of the kind 
start s by V start- Denote the auxiliary actions of C" that occur between a start action and a 
state »j by V\, and denote the auxiliary actions of C" that occur between a state *j and the 
corresponding occurrence of action aj by Vi- 

Let V be a trace distribution of Mi||C that is not a trace distribution of M2IIC. Consider 
a probabilistic execution H\ of Mi||C whose trace distribution is V in Mi||C, and consider the 
scheduler that leads to H\ in Mi||C. Apply to Mi||C" the same scheduler with the following 
modification: whenever some transition of C is scheduled, schedule the start action from C", 
then schedule the internal transitions to choose the transition of C to perform with the right 
probability, and then schedule the transitions of the chosen transition till the corresponding 
external action of C occurs. Denote the resulting probabilistic execution by H[ and the resulting 
trace distribution by V . Then, 



V \ acts(M 1 \\C) = V. 



(7.43) 



To prove (7.43), we define a new construction, called shrink and abbreviated with shr, to be 
applied to probabilistic executions of M 8 ||C" such that no action of M 8 - occurs between a state 
of the form *j and the occurrence of the corresponding action aj of C, and such that all the 
transitions between a state of the kind *j and the corresponding occurrences of action aj are 
scheduled. 

Let H' be such a probabilistic execution of M 8 ||C". Denote shr(H') by if . A state q of H' 
is closed if each occurrence of a state of the kind *j is followed eventually by the occurrence of 
the corresponding action aj. For each state q of H' let shr(q) be obtained from q as follows: 
each sequence 

(s , c ) start Co ( s , •)b 1 (s 1 , •) • • -b h {s h , •j)T 1 (s h , •) • • -T k {s h , •)a J (s, c) 
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is replaced with 

(s ,c )b tl (s tl ,c ) ■ ■ ■b tl (s h i,c )a J (s,c), 

where i\ , . . . , i\ is the ordered sequence of the indexes of the 6's that are actions of Mi, and each 
sequence either of the form 

(s , c ). start Co ( s , •)b 1 (s 1 , •) • • -b h {s h , • J )T 1 (s h , •) • • -T k {s h , •) 
or of the form 

(s , c ). start Co ( s , •)b 1 (s 1 ,») ■ ■ -b h {s h , •) 
occurring at the end of q is replaced with 

(s ,c )b tl (s tl ,c ) • • -b tl (s h i,c ), 
where i\, . . . ,i\ is the ordered sequence of the indexes of the 6's that are actions of M 8 -. Then, 

states(H) = {shr(q) \ q £ states(H')}. (7.44) 

Let {q,V) be a restricted transition of H' , and suppose that no action of acts(C')\acts(C) 
occurs. Let 0' = {(a,shr(q r )) \ (a,q r ) £ S7}, and for each (a,q") £ S7', let P'[(a,g")] = 
P[a X s/jr _1 (g")], where s/ir _1 (g) is the set of states g' of H' such that shr(q') = q. Then the 
transition shr((q,V)) is defined to be 

shr((q,V)) = (shr(q),V). (7.45) 

For the transition relation of H , consider a state q of H , and let ram( shr~ x (qj) be the set of 
minimal states of shr~ (q) under prefix ordering. For each state q £ shr~ (q), let 

The transition enabled from q in H is 

E P? r " 1(9) ^'[« cfe ( M «ll C )] s M^' r acts{Mi\\C)). (7.47) 

q'(zshr~ (g) 

The probabilistic execution if satisfies the following properties. 

a. if is a probabilistic execution of M 8 ||C. 

The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of if is a finite execution fragment of M 8 ||C follows from a simple 
analysis of the definition of shr. 

We need to show that for each state q of H the transition of Expression (7.47) is generated 
by a combination of transitions of M 8 ||C. The states of s/ir _1 (g) that enable some action 
of Mj-||C can be partitioned into two sets C and O of closed and open states, respectively. 

We analyze C first. Let q' £ C . Since tr q t is a transition of H' , (tr q i \ acts(Mi\\C)) can 
be expressed as J2j PjW ~ ^ r j)? where each trj is a transition of M 8 ||C". We distinguish 
two cases. 
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1. tr-j is a transition of M 8 -. 

Then tr-j = ((s,c),a,V ® D(c)) for some action a and probability space V, where 
(s,c) = Istate(q'). Let lstate(shr(q')) = (s',c'). Then, s' = s, as it follows directly 
from the definition of shr. Moreover, (s,a,V) is a transition of M 8 -. Define tr 1 - 
to be the transition ((s,c'),a,V ® V(c')). Then tr'- is a transition of M 8 ||C and 
shr q (q'~ tr{) = q~ tr'-. 

2. tr-j is a transition of C". 

This case is not possible since, from the construction of C", no action of C can be 
enabled from a closed state. 

Observe that shr distributes over combination of transitions. Thus, 

shr(trf,' \ acts(Mi\\C)) = Y^Pj( shr (tf) ~ * r j)> ( 7 - 48 ) 

3 

which is generated by a combination of transitions of M 8 ||C. 

We now turn to O . The set O can be partitioned into sets (®j)j>o, where each set 
Qj consists of those states q' of O where a particular state *j of C" occurs without its 
matching action cij. Each element q' of Qj can be split into two parts q\ ~ q^t where 
lstate(qi)\C" = »j. Denote q\ by head(q'). Partition Qj into other sets (0j,fc)fc>o 5 where 
each Qj t k is an equivalence class of the relation that relates two states iff they have the 
same head. Denote the common head of the states of 8J - by head(Qij). For each pair 
of states qi,q2 of H' such that q\ < q^t denote by p qiq2 the probability value such that 
Ph'[CE ] = Ph'[Cc? ]Pq 1 q 2 - Then, for each equivalence class ©;j, the expression 

J2 p s f r ~ 1{9) PF'[acts(Mi\\C)]shr(tr$' \ acts(Mi\\C)) (7.49) 

q'e&j, k 

can be rewritten into 

(_s/tr _1 (g) V^ \ 

Phead(@ij) Z^ Phead(q')q' I 

£ v Pheadiq ' W P^'[a 3 ]shr(trf \ acts(M % \\C)) (7.50) 

q'e& 3tk ^<l'£ @ j,k Phead(q')q' 

where (7.50) is obtained from (7.49) by expressing each f/ q Y {q) as pf^ d ^Phead{q') q h by 
grouping p\ e [ i( Q t y which is equal to P^^V) for each q' os 8J , by substituting Pfi'[a,j] 
for P^f [acts(Mi\\C] (action cij is the only action of M 8 ||C that can be performed from q' 
due to the structure of H'), and by multiplying and dividing by J2 q 'e& ■ k Phead(q')q'- 

Observe that each transition that appears in (7.50) is generated by some transitions of 
Mj-||C. Thus, the transition of (7.50) is generated by a combined transition of M 8 ||C. 
Denote this transition by tr-j^. Then, in Expression (7.47) it is possible to substi- 
tute each subexpression J2 q 'e@ P' ' ^ P^f [acts(Mi\\C)]shr(tr q i \ acts(Mi\\Cj) with 
(/hladiq'l^q'ee^kPheadiq^tr^k- This is enough to conclude. 
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b. For each state q of H , 

P H [C q ]= £ P H ,[C q ,]. (7.51) 

q' (zTtiin(shr~ (g)) 

This is shown by induction on the length of q. If q consists of a start state only, then the 
result is trivial. Otherwise, from the definition of the probability of a cone and (7.47), 

P H [C qas }= J2 PH\C q ,]P^[axshr-\qas)]. (7.52) 

q'(zshr~ (g) 

Observe that the states of min( shr -1 (qasj) are the states that appear in (ax shr -1 (qasj)f] 
tt q , for some q' £ shr^iq). Thus, P H [C qas ] = E 3 ' em i„(,A r -i(g as )) p H'[C q >]. 

c. tdistr(H) = tdistr(H') \ acts(M t \\C). 

Let (3 be a finite trace of if or the projection of a finite trace of H' . Then {a £ 0#/ | /3 < 
trace(a) \ acts(Mi\\C)} can be expressed as a union of disjoint cones U ge eCg where 

= {q e states(H') \ trace(q) \ acts(M t \\C) = /3, lact(q) = lact{fi)}. (7.53) 

Observe that is a set of closed states. The set shr(Q) is the set 

shr(Q) = {q £ states(H) \ trace(q) = (3, lact(q) = lact(P)}, (7-54) 

which is a characterization of {a £ S7# | (3 < trace(a)} as a union of disjoint cones. 
Observe that min( shr -1 ( shr(Q)j) = 0. Moreover, for each q\ ^ q^ of s/ir(0), shr~ 1 (qi)f] 
shr- 1 (q 2 ) = 0. Thus, from (7.51), P w [U qe@ C q ] = P H [q £ shr(&)C q ]. 

To complete the proof of (7.43), it is enough to observe that H\ = shr(H[). Property (7.43) is 
then expressed by property (c). 

Suppose by contradiction that it is possible to obtain V from M2IIC". Consider the scheduler 
that leads to V in M2IIC", and let H' 2 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution H" of M2IIC" whose trace distribution is V , such that there 
is no action of M 2 between each state of the kind •; and the occurrence of the corresponding 
external action of C, and such that all the transitions between a state of the kind »j and the 
corresponding occurrences of action cij are scheduled. Then we let H 2 = shr(H 2 ). This leads 
to a contradiction since tdistr(H 2 ) = V. The rest of the proof is dedicated to the construction 
of H'l 

For each state q of H' 2 , let shf(q) be the set of sequences q' that can be obtained from q as 
follows: each sequence 

(s , • ? )&i(si, •) • • -bkisk, •)« J (s, c) 

is replaced with 

(s , •j)b il (s , •)■■■ b H (s , •jcijiso, c)b kl (s kl , c) • • -b km {s, c) 
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where ii, . . . , i; is the ordered sequence of the indexes of the 6's that are actions of C", and 
k\, . . . , k m is the ordered sequence of the indexes of the 6's that are actions of M2; each sequence 

(so,»j)b 1 (s 1 ,»)---b k (s k ,») 

occurring at the end of q either is replaced with 

(s , •j)b il (s , •) • ■■b H (s , •) ~ a ~ (s , •)a J (s , c)6 fcl (s fcl , c) • • -6jt m (s, c) 

where ii, . . ., i\ is the ordered sequence of the indexes of the 6's that are actions of C", k\, . . . , k m 
is the ordered sequence of the indexes of the 6's that are actions of M2, and a, called an extension 
for g, is an arbitrary execution fragment of M2IIC" that leads to the occurrence of a,j, or, is 
replaced with a prefix of (so, •j)fr; 1 (so, •) • • -^(so, •). Then, 

states{H' 2 r ) = \J shf(q). (7.55) 

Let (g, V) be a restricted transition of H 2 , and suppose that only actions of Mi and V s t ar t occur. 
Let q' be a state of shf(q). Then, for each (a, #1) £ there is exactly one g x £ shf(qi) such that 
<?' < <?i an( i kil = W\ + 1- Denote such q[ by shf ,(q\). Let 0' = {(a, shf q i(qi) \ (a,qi) £ 0}, 
and let, for each (a, g x ) £ 0', i"[(a, g x )] = -P[(« X s/i/ _1 (g 1 ))], where s/i/ _1 (g) is the set of states 
g' of H 2 such that g £ shf(q'). Then define the transition shf ,((q,V)) to be 

shf q ,((q,V)) = (q',V). (7.56) 

For each state g of if", let min(shf~ (qj) be the set of minimal states of shf~ (q) under prefix 
ordering. Let q be a closed state of if", and let q' £ s/i/ _1 (g). If g' is an open state, then let a 
be the extension for g' that is used in g, and let E q , be the product of the probabilities of the 
edges of a. For each state q' of s/j/ _1 (g), where g is closed, let 

• V\> = P H^[C q >] if q' is closed; 

• P Q q> = P H!,[C q i]E q q , if q' is open. 
For each g' £ s/j/ _1 (g), let 



P*-"' S i (7.57) 

^ 3 "eili.«(s)l/- 1 ( g ))f'g" 

If g is open, then the transition enabled from g in if" is the one due to the transition of C" 
enabled from lstate(q)\C"; if g is closed, then the transition enabled from g in if" is 

E P? /_1(9) ^[«cfo(if^\(acfo(C) U V 2 )} (7.58) 

^/,(<V* r (acts(H' 2 )\(acts(C) U V 2 ))). 
The probabilistic execution H 2 satisfies the following properties. 
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a. H'2 is a probabilistic execution of M2IIC". 

The fact that each state of H" is reachable can be shown by a simple inductive argument; 
the fact that each state of R' 2 ' is a finite execution fragment of M2IIC" follows from a 
simple analysis of the definition of shf. 

We need to check that for each state q of if" the transition enabled from q in if" is 
generated by a combination of transitions of M2IIC". If q is an open state, then the result 
follows immediately from the definition of the transition relation of if"- If g is a closed 

TTl 

state, then consider a state q' £ shf~ (q). The transition tr , 2 \ (acts(H' 2 )\V2)-, which 
appears in Expression (7.58), can be expressed as 2iP«(<?'~ ^ r i)i where each tr-{ is a 
transition of M2IIC" enabled from Istate(q'). We distinguish two cases. 

1. tri is a transition of M 2 . 

Then tr-{ = ((s,c),a,V ® D(c)) for some action a and probability space V, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s' = s. Define tr\ to be the 
transition ((s,c'),a,V®V(c 1 )). Then tr\ is a transition of M2IIC" and shf Jq'^ tr{) = 
q~ tr\. 

2. tri is a transition of C". 

Then tr-{ = ((s,c),a,V(s) (x) V) for some action a and probability space V, where 
(s,c) = Istate(q'). Let Istate(q) = (s',c'). Then, s' = s and c = c' (q is closed). 
Define fr'- to be fr,-. Then tr\ is a transition of M2IIC" and shf Jq' ^ tr{) = q ^ tr\. 

Observe that shf distributes over combination of transitions, and thus, the transition 

TTl 

shf (t i 2 \ (acts(H'2)\V2)) can be expressed as J^iPiiQ^ ^')' which is generated by a 
combination of transitions of M2IIC". 

b. For each state q of if", 

P H »[C q ] = I ^^ W -^ |f ? - closed, 

21 qi 1 E rt ™( Jr >(,))^[^] if g is open. 

The proof is by induction on the length of q. If q consists of a start state only, then the 
result is trivial. Otherwise, consider PH"[C qas \- We distinguish two cases. 

1. q is open. 

In this case a is an action of V2 U acts(C), and each state of shf~ (q) is open. From 
the definition of the probability of a cone and induction, 

P H !>[C q as}=[ E ^[^'1 KM(«.H- (7-60) 

VemmfstJ-'f,)) / 

We distinguish two other cases. 

(a) a e V 2 . 

Observe that all the states of mm(s/i/ _1 (g)) enable the same transition of C 
that is enabled from q. Moreover, for each q' £ min(shf~ (<?)), action a occurs 
with probability 1 (in V each occurrence of a start action is followed by an 
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external action with probability 1), and the probability of reaching a state of 

H" 
min(shf~ (qas)) given that a occurs is P q 2 [(a, qas)] (recall that q enables only 

action a). Since all the states of min(shf~ (qas)) are open and have a prefix in 

min(shf~ 1 (q)), we can conclude 

P H »[C q as]= E Ph&CA- ( 7 - 61 ) 

q'(zTtiin(shf~ (qas)) 

(b) a £ acts(C). 

H" 
From the definition of if", Pq 2 [(a, qas)] = 1. Observe that all the states of 

min(shf~ 1 (qj) enable the same transition of C that is enabled from q. Moreover, 

for each q' £ min(shf~ 1 (q)), action a occurs with probability 1 (in V each 

occurrence of a start action is followed by an external action with probability 

1), leading to a state of shf -1 (qas) for sure (recall that q enables only action a). 

Thus, for each q' £ shf~ (q), 

P H ' 2 [Cq']= J2 P H >[Cq»}. (7.62) 

q" £min(shf~ (qas^lq'-Cq" 

Combining (7.60) and (7.62), we obtain 

P H »[C q as}= E P^[Cq'}. (7.63) 

q'(zTtiin(shf~ (qas)) 

For each q' £ min(shf~ (gas)), if q' is open, then p q T = Pn'iCq'] by definition; 
if q' is closed, then p q ? s = P H * [C q i] since E q ? s = 1 (no a must be added by shf 
to get q' from qas). Thus, (7.63) becomes 

PH>>[c qa .]= E pT- ( 7 - 64 ) 

q'(zTtiin(shf~ (qas)) 

2. q is closed. 

In this case, from the definition of the probability of a cone and (7.58), 

P H n[C qas ] = P H ,,[C q ] l E P? /_1(9) P5[a X shf-Hqas)]] (7.65) 

Xq'eshf-^q) J 

From induction, the definition of p s , 9 , and an algebraic simplification, 

P H »[C qa s] = E P H i[C q ,]P q H i[axshf- 1 (qas)]+ (7.66) 

g'Gs/t/- 1 (g)| C ;ose(i(g') 

E P H >[C q .]E q ql rf*[a X shf- 1 (qas)]. 

q'eshf~ 1 (q)\o-pen(q') 

We distinguish two subcases. 

(a) qas is open. 

In this case each state q' of shf~ (q) such that Pi 2 [a X shf~ (qas)] > is 
closed, and thus only the first summand of (7.66) is used. Moreover, for each q' 

of shf~ (q) the set Q, 2 n a X shf~ (qas) is made of open states q'as' such that 
Eq^ s , = 1. Observe that all the states of min(shf~ (qas)) are captured. Thus, 

PH>>[C q as]= E P\- ( 7 - 67 ) 

q , £min{shf~ (qas)) 
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(b) qas is closed. 

In this case, for each q' £ s/j/ _1 (g), if q' is closed, then all the states reached in 
£l q i fl ({a} X shf~ (qas)) are closed, and if q' is open, then all the states reached 
in fl q i fl ({a} X shf~ 1 (qasj) are open and the extension a does not change, i.e., 
the term E does not change. Observe that all the states of min(shf~ 1 (qasj) are 
captured. Thus, 

PH»[C q as]= E P\- ( 7 - 68 ) 

q f £min(shf~ (qas)) 

c. tdistr(H' 2 ) = tdistr(H%). 

Let fi be a finite trace of i/^ or iZ"- Then {a £ Cl H i \ fi < trace(a)} can be expressed as 
a union of disjoint cones U ge eCg- We distinguish two cases. 

1. fi does not end with an action of C . 
Then 

= {q £ states(H') \ trace(q) = fi, lact(q) = lact(fi)}. (7.69) 

The set 0' = {q £ shf(Q) \ lact(q) = lact(fi)} is a characterization of {a £ Q, H n \ 
fi < trace(a)} as a union of disjoint cones. Observe that min(shf~ 1 (Q'j) = and 
that for each q\ ^ q^ of 0', min(shf~ (qij)r\min(shf~ (^2)) = 0- Thus, from (7.51), 
Ph^W e tt H > I P < trace(a)}] = P H »[{a £ tt H ,, \ /3 < trace(a)}]. 

2. /3 ends with an action of C . 

In this case fi = (3'aj for some action aj £ acts(C). Since in i/^ and if" after the 
occurrence of a state «j the corresponding action aj occurs with probability 1, we 
can assume that all the states of end in »j, i.e., 

= {q £ states(H') \ trace(q) = fi 1 ', and Istate(q) is one of the "j's}. (7.70) 

Then the set 0' = min(shf(Q)) is a characterization of {a £ 0#// | fi < trace(a)} as a 
union of disjoint cones. Observe that ah the elements of are open. Property (7.59) 
is sufficient to conclude. ■ 

Lemma 7.5.7 Let C be a distinguishing binary separated context for two probabilistic automata 
Mi and Mi- Then there exists a distinguishing total binary separated context C" for Mi and 
Mi where all the probabilistic transitions have a uniform distribution. C" is called a balanced 
separated context. 

Proof. We achieve the result in two steps. First we decompose a binary probabilistic transition 
into several binary uniform probabilistic transitions, leading to a new distinguishing context 
C\; then we use Lemma 7.5.4 to make C\ into a cycle-free context. 

The context C\ is obtained from C by expressing each probabilistic transition of C by 
means of, possibly infinitely many, binary probabilistic transitions. For each state s of C, let 
start s be a new action. If s enables a probabilistic transition with actions a\, a^ to states si, S2, 
respectively, and with probabilities pi,P2, respectively, then C\ enables from s a deterministic 
transition with action start s . Then, C\ enables an internal probabilistic transition with a 
uniform distribution. If pi > pi (p? > Pi), then one of the states that is reached enables a 
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deterministic transition with action a\ (o^)- The other state enables a new internal probabilistic 
transition with a uniform binary distribution, and the transitions from the successive states are 
determined by giving a\ probability 2(pi — 1/2) and a^ probability 2p2 («i probability 2pi 
and a2 probability 2(p2 — 1/2))- If Pi = P2 5 then one state enables a\, and the other state 
enables a^- For example, if pi = 5/8 and pi = 3/8, then the corresponding transitions of C\ 
are represented below. Let V be a trace distribution of Mi||C that is not a trace distribution 




of M2HC. Consider a probabilistic execution H\ of Mi||C whose trace distribution is V, and 
consider the scheduler that leads to Hi in Mi||C. Apply to Mi||Ci the same scheduler with 
the following modification: whenever a probabilistic transition of C is scheduled, schedule the 
start action from C\, then schedule the internal transitions to resolve the probabilistic choice, 
and finally schedule the chosen action. Denote the resulting probabilistic execution by H[ and 
the resulting trace distribution by V . Then, 



V \ acts(M l \\C) = V. 



f7.71) 



To prove (7.71), we define a new construction shr\, similar to shr, to be applied to probabilistic 
executions of M 8 ||Ci such that no action of M 8 - occurs between the occurrence of a start s action 
and the occurrence of one of the corresponding external actions of C, and such that all the 
transitions of C\ between the occurrence of an action start s and the occurrence of one of the 
corresponding external actions of C are scheduled. The new function is identical to shr if we 
consider each state reached immediately after the occurrence of a start action like the states »j 
used in Lemma 7.5.6. We leave the details to the reader. 

Suppose by contradiction that it is possible to obtain V from M2HC1. Consider the scheduler 
that leads to V in M2HC1, and let H' 2 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution H'<{ of M2HC1 whose trace distribution is V' , such that 
no action of M 8 - occurs between the occurrence of a start s action and the occurrence of one 
of the corresponding external action of C, and such that all the transitions of C\ between 
the occurrence of an action start s and the occurrence of one of the corresponding external 
action of C are scheduled. Then we let if 2 = shri(H"). This leads to a contradiction since 
tdistr(H 2 ) = V. 

The construction of if", which is left to the reader, is the same as shf if we consider each 
state reached immediately after the occurrence of a start action like the states »j used in 
Lemma 7.5.6. ■ 

Lemma 7.5.8 Let C be a distinguishing balanced separated context for two probabilistic au- 
tomata Mi and Mi- Then there exists a distinguishing binary separated context C" for Mi 
and Mi with no internal actions and such that each action appears exactly in one edge of the 
transition tree. C" is called a total balanced separated context. 
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Proof. The context C" is obtained from C by renaming all of its actions so that each edge of 
the new transition relation has its own action. 

Let V be a trace distribution of Mi||C that is not a trace distribution of M2IIC. Consider a 
probabilistic execution H\ of Mi||C whose trace distribution is V, and consider the scheduler 
that leads to Hi in Mi||C. Apply to Mi||C" the same scheduler with the following modification: 
whenever a transition of C is scheduled, schedule the corresponding transition of C" . Denote the 
resulting probabilistic execution by H[ and the corresponding trace distribution by V . From 
construction, Hi and H[ are the same up to the names of the actions of C . Thus, if p' is the 
function that maps each action of C" to its original name in C, V = p'{V) (the renaming of a 
trace distribution is the probability space induced by the function that renames traces). 

Suppose by contradiction that it is possible to obtain V from M2IIC". Consider the scheduler 
that leads to V in M2IIC", and let H 2 be the corresponding probabilistic execution. Apply to 
M2HC the same scheduler with the following modifications: whenever a transition of C" is 
scheduled, schedule the corresponding transition of C with the unrenamed actions. Let H 2 be 
the resulting probabilistic execution. From the construction, H 2 and H 2 are the same up to 
the names of the actions of C . Thus, tdistr(H 2 ) = p'{P') = ^? which is a contradiction. ■ 

Lemma 7.5.9 Let C be a distinguishing total balanced separated context for two probabilistic 
automata Mi and M 2 . Then there exists a distinguishing total balanced separated context C" 
for Mi and M 2 that from every state enables two deterministic transitions and a probabilistic 
transition with a uniform distribution over two choices. C" is called a complete context. 

Proof. In this case it is enough to complete C by adding all the missing transitions and states. 
If V is a trace distribution of Mi||C that is not a trace distribution of M2IIC, then it is enough 
to use on Mi||C" the same scheduler that is used in Mi||C. In fact, since each new transition 
of C" has a distinct action, none of the new transitions of C" can be used in M2IIC" to generate 
V. m 

Lemma 7.5.10 Let C be a distinguishing complete context for two probabilistic automata Mi 
and M 2 . Then the principal context Cp is a distinguishing context for Mi and M 2 . 

Proof. The result is achieved in two steps. First the actions of C are renamed so that each state 
enables two deterministic transitions with actions left and right, respectively, and a probabilistic 
transition with actions pleft and pright. Call this context C\. Then, by observing that each 
state s of C\ is uniquely determined by the trace of the unique execution of C\ that leads to s, 
all the states of C\ are collapsed into a unique one. 

Thus, we need to show only that C\ is a distinguishing context. Let Pbea trace distribution 
of Mi||C that is not a trace distribution of M2IIC. Consider the scheduler that leads to V in 
Mi||C, and apply to Mi||Ci the same scheduler with the following modification: whenever a 
transition of C is scheduled, schedule the corresponding transition of C\. Denote the resulting 
trace distribution by V . Note that if we rename all the actions of C\ into their original name 
in C, then we obtain V. 

Suppose by contradiction that it is possible to obtain V from M2HC1. Consider the sched- 
uler that leads to V in M2HC1, and apply to M2HC the same scheduler with the following 
modification: whenever a transition of C\ is scheduled, schedule the corresponding transition 
of C . The resulting trace distribution is V, which is a contradiction. ■ 
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Lemma 7.5.11 Let Cp be a distinguishing context for two probabilistic automata M\ and M 2 . 
Then the simple principal context, denoted by C, is a distinguishing context for Mi and M 2 . 

Proof. Let V be a trace distribution of Mi||Cp that is not a trace distribution of M2\\Cp. 
Consider a probabilistic execution Hi of Mi||Cp whose trace distribution is V, and consider 
the scheduler that leads to Hi in Mi||Cp. Apply to Mi||C the same scheduler with the follow- 
ing modification: whenever the probabilistic transition of Cp is scheduled, schedule the start 
action of C followed by the next transition of C that becomes enabled. Denote the resulting 
probabilistic execution by H[ and the resulting trace distribution by V . Then, 

V \ acte(Mj || Cp) = V. (7.72) 

To prove (7.72), we define a new construction shr 2 , similar to shr, to be applied to probabilistic 
executions of M 8 ||C such that no action of M 8 - occurs between the occurrence of a start action 
and the occurrence of one of the actions pleft and pright, and such that the transitions labeled 
with pleft and pright occur whenever they are enabled. The new function is identical to shr 
if we consider each state reached after an action start as a state of the kind »j. We leave the 
details to the reader. 

Suppose by contradiction that it is possible to obtain V from M2IIC. Consider the scheduler 
that leads to V in M2IIC, and let H 2 be the corresponding probabilistic execution. First, we 
build a new probabilistic execution H 2 of M2HC whose trace distribution is V , such that no 
action of M 2 occurs between the occurrence of a start action and the occurrence of one of 
the actions pleft and pright, and such that the transitions labeled with pleft and pright occur 
whenever they are enabled. Then we let H 2 = clp 2 (H 2 ). This leads to a contradiction since 
tdistr(H 2 ) = V. 

The construction of H 2 , which is left to the reader, is the same as shf if we consider each 
state reached immediately after the occurrence of a start action like the states »j used in 
Lemma 7.5.6. ■ 

Proof of Theorem 7.5.1. Let M x Q DC M 2 . Then, from Lemma 7.5.11, Mi\\C P Q D M 2 \\C P . 
Conversely, let Mi\\C P Q D M 2 \\C P . Then, from Lemmas 7.5.3, 7.5.4, 7.5.5, 7.5.6, 7.5.7, 7.5.8, 
7.5.9, and 7.5.10, M x Q DC M 2 . ■ 

7.6 Discussion 

A trace-based semantics similar to ours is studied for generative processes by Jou and Smolka 
[JS90]. One of the processes of Jou and Smolka is essentially one of our probabilistic executions. 
The semantics of a process is given by a function, called a trace function, that associates a prob- 
ability with each finite trace. Since our trace distributions are determined by the probabilities 
of the cones, our trace distributions are characterized completely by the trace functions of Jou 
and Smolka. In other words, the trace semantics of Jou and Smolka is the semantics that we 
use to say that two probabilistic executions have the same trace distribution. 

Jou and Smolka define also a notion of a maximal trace function. Given a probabilistic 
execution H , the interpretation of a maximal trace function in our framework is a function that 
associates with each finite trace (3 the probability of the extended executions on 0# that end in 
8 and whose trace is (3. Jou and Smolka show that the trace function of a process is sufficient 
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to determine the maximal trace function of the process. In our trace distributions the maximal 
trace function of a probabilistic execution is given by the probability of each finite trace in the 
corresponding trace distribution. From the definition of a trace distribution the probability of 
each finite trace is determined uniquely by the probabilities of the cones, and thus the result of 
Jou and Smolka holds also in our framework. 
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Chapter 8 

Hierarchical Verication 
Simulations 

8.1 Introduction 

In Chapter 7 we have studied the trace distribution precongruence as an instance of the hierar- 
chical method for the verification of probabilistic systems. Another instance of the hierarchical 
method is called the simulation method. According to the simulation method, rather than 
comparing two probabilistic automata through some abstract observations, two probabilistic 
automata are compared by establishing some relation between their states and by showing that 
the two probabilistic automata can simulate each other via the given relation. Standard work 
on simulation relations appears in [Mil89, Jon91, LV91]. Simulation relations are stronger than 
the trace preorder, and are often used as a sound proof technique for the trace preorder. 

In this chapter we study how to extend some of the relations of [Mil89, Jon91, LV91] to the 
probabilistic framework. We start with the generalization of the simplest relations that do not 
abstract from internal computation, and we conclude with the generalization of the forward 
simulations of [LV91] that approximate closely the trace distribution preorder. We prove the 
equivalent of the Execution Correspondence Lemma [GSSL94] for probabilistic automata, which 
states that there is a strong connection between the probabilistic executions of two probabilistic 
automata related by some simulation relation. Finally, we use the new Execution Correspon- 
dence Lemma to prove that the existence of a probabilistic forward simulation is sufficient to 
prove the trace distribution precongruence relation. 

8.2 Strong Simulations 

One of the finest equivalence relations for ordinary automata would be graph isomorphism; 
however, it is widely recognized that graph isomorphism distinguishes too much. A coarser 
equivalence relation is strong bisimulation [Par81, Mil89], where two automata A\ and A 2 are 
equivalent iff there is an equivalence relation between their states so that for each pair (si,S2) 
of equivalent states, 

if s\ — ► s^, then there exists a state s 2 equivalent to s^ such that s 2 — ► s 2 . 
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Figure 8-1: The difference between strong bisimulation and the kernel of strong simulation. 

That is, A\ and A 2 simulate each other. A preorder relation that is closely connected to 
strong bisimulation is strong simulation. An automaton A\ is strongly simulated by another 
automaton A 2 iff there is a relation between the states of A\ and the states of A 2 so that for 
each pair (si,S2) of related states, 

if si — > s'-l, then there exists a state s 2 such that s 2 — ► s 2 and s^ is related to s' 2 . 

The kernel of strong simulation is an equivalence relation that is coarser than bisimulation. 

Example 8.2.1 (Strong simulation and strong bisimulation) Figure 8-1 shows the dif- 
ference between strong bisimulation and the kernel of strong simulation. The double-arrow 
dotted links represent a strong bisimulation between A\ and A 2 ; thus, A\ and A 2 are strongly 
bisimilar. There is also a strong simulation from A 2 to A3, expressed by the dotted lines that 
have an arrow pointing to A3, and a strong simulation from A3 to A 2 , expressed by the dotted 
lines that have an arrow pointing to A 2 . Thus, A 2 and A3 are equivalent according to the kernel 
of strong simulation. However, there is no bisimulation between A 2 and A3 since state s 2 of A3 
must be related to state si of A 2 in order for A 2 to be able to simulate the transition so — > s 2 
of A3, but then it is not possible to simulate the transition si — > S3 of A 2 from s 2 in A3. ■ 

The extension of strong bisimulation and strong simulation to the probabilistic framework 
presents a problem due to the fact that a probabilistic transition leads to a probability distri- 
bution over states rather than to a single state. Thus, a relation over states needs to be lifted 
to distributions over states. Here we borrow an idea from [JL91]. 

Let 1ZC X X Y be a relation between two sets X,Y, and let V\ and V2 be two probability 
spaces of Probs(X) and Probs(Y), respectively. Then V\ and V2 are in relation Q-ji, written 
V\ Ck V2, iff there exists a weight function i»:lxy^[0,l] such that 

1. for each x G X , J2 y eY w ( x , y) = -Pif^L 

2. for each y G Y, J2 xe x w i x , y) = p 2[y], 

3. for each (x, y) G X X Y , if w(x, y) > then x 1Z y. 

Example 8.2.2 (Lifting of one relation) The idea behind the definition of C^ is that each 
state of Oi must be represented by some states of CI?, and similarly, each state of O2 must 
represent one or more states of Oi . Figure 8-2 gives an example of two probability spaces that 
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Figure 8-2: Lifting one relation. 

are related. The dotted lines connect states that are related by 1Z. Thus, state s\ t i can be 
represented by S2,i for a third of its probability, and by s 2j2 f° r the reminder. Similarly, state 
s 2,2 represents s\ t i for one sixth of its probability and si^ for the reminder. A useful property 
of C^ is its preservation over combination of probability spaces. ■ 

If 1Z is an equivalence relation, then we denote the relation C^ alternatively by =-jz- The reason 
for the alternative notation is that whenever 1Z is an equivalence relation and V\ =-jz V 2 , each 
equivalence class of 1Z is assigned the same probability in V\ and V 2 (cf. Lemma 8.2.2). 

The definition of strong bisimulation and strong simulation for probabilistic automata are 
now straightforward. For convenience assume that M\ and Mi do not have common states. 
A strong bisimulation between two simple probabilistic automata M\,M 2 is an equivalence 
relation 1Z over states(M\) U states(M 2 ) such that 

1. each start state of Mi is related to at least one start state of M2, and vice versa; 

2. for each pair of states si 1Z s 2 and each transition si — ► V\ of either M\ or M2, there 
exists a transition s 2 — > V 2 of either M\ or Mi such that V\ =-jz V 2 . 

We write M\ ~ Mi whenever acts(M\) = acts^M?) and there is a strong bisimulation between 
Mi and M 2 . 

A strong simulation between two simple probabilistic automata M\,Mi is a relation 1ZC 
states(M\) X states^M?) such that 

1. each start state of Mi is related to at least one start state of M2; 

2. for each pair of states si 1Z si and each transition si — ► V\ of Mi, there exists a transition 
s 2 --> V 2 of M 2 such that V\ Qn V 2 - 

We write Mi Cgg M2 whenever acts(M\) = acts^M?) and there is a strong simulation from Mi 
to M 2 . We denote he kernel of strong simulation by =ss- Because of Lemma 8.2.2, our strong 
bisimulations are the same as the bisimulations of [Han94], and our strong simulations are a 
generalization of the simulations of [JL91]. 

It is easy to check that ~ is an equivalence relation, that Cgg is a preorder relation, and 
that both ~ and Cgg are preserved by the parallel composition operator. 

We conclude this section by proving two results about the lifting of a relation. The first 
result shows that the lifting of a relation is preserved by the combination of probability spaces; 
the second result shows that V\ =-jz V 2 iff V\ and V 2 assign the same probability to each 
equivalence class of 1Z. 
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Lemma 8.2.1 Let Vx,i Ek Vy,i via a weight function W{, and let {pi}i>o be a family of 
real numbers between and 1 such that J2i>oPi = 1- Then Y^i>oViPx,i Ek Eix^'^V,*' v ^ a 

Proof. Let V x = Y,i>oPiPx,i, V Y = E;>oP;"£V,;, and w = Y^i>oPi w i- Let x € & x . Then 

J2 ye u Y w ( x ,y) = J2 ye uY T,i>oPiM x ,y) = J2 t >oPi(12yen Y w i( x ,v)) = J2 t >oP*PxA x ] = Px[x]. 
Condition 2 of the definition of C^ is verified similarly. For Condition 3, let w(x,y) > 0. Then 
there exists an i such that Wi(x, y) > 0, and thus x 1Z y. ■ 

Lemma 8.2.2 Let X,Y be two disjoint sets, 1Z be an equivalence relation on X U Y , and let 
V\ and V 2 be probability spaces of Probs(X) and Probs(Y), respectively. Then, V\ =-jz V 2 iff 
for each equivalence class C of (X U Y)/U, Pi[C l~l fii] = P 2 [C l~l fi 2 ]- 

Proof. Suppose that V\ =-jz V 2 , and let w be the corresponding weight function. Then, for 
each equivalence class C of (X U Y)/1Z, 

p 1 [cnn 1 ]= E p iW= E E w (^), (8.1) 

iGCnfii iGCnfii yeCnQ 2 

and 

p 2 [cnn 2 ]= e p M= E E w ( x >y)- ( 8 - 2 ) 

From the commutativity and associativity of sum, 

Pi[Cnfii] = p 2 [Cnfi 2 ]. (8.3) 

Conversely, suppose that each equivalence class (X U Y)/1Z has the same probability in V\ and 
P 2 . We define w(x,y) for each equivalence class of (X U Y)/1Z, and we assume implicitly that 
w is for all the pairs (x,y) G Oi X i7 2 that are not considered in the construction below. 
Consider any equivalence class C of (X U Y)/1Z, and let X' = C P\ Oi, and Y' = C P\ i7 2 . From 
hypothesis we know that Pi[X'] = PjfY']. Let x\,x 2 , ... be an enumeration of the points of 
X', and let y\,y 2 , ••• be an enumeration of the points of Y'. For each i, let pi = Efc<« Pi[ x i] 
and let q t = Efc<; -^2 [?/;]• Then 

w( Xi y) = { ° lf Pt+1 ~ qj ° r qj+1 ~ Pt 

■^ 1 min(pi + i,qj + i) — max(pi,qj) otherwise. 

Informally, the construction above works as follows. Consider two intervals [0,Pi[X']], and 
mark the first interval with the points pi and the second interval with the points qj. Each 
interval [p 8 ,p 8 _|_i] has length Pi [2;;] and each interval [gj,gj + i] has length P 2 [yj]- The weight 
function w(xi, yj) is defined to be the length of the intersection of the intervals associated with 
Xi and yj, respectively. Lt is simple to verify that w is a weight function for V\ and P 2 . ■ 
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Figure 8-3: Combining transitions to simulate a transition. 

8.3 Strong Probabilistic Simulations 

In the definition of strong bisimulations and strong simulations we have not taken into account 
the fact that the nondeterminism can be resolved by combining several transitions probabilis- 
tically into a unique one. That is, a transition of a probabilistic automaton could be simulated 
by combining several transitions of another probabilistic automaton. 

Example 8.3.1 (Combining transitions to simulate another transition) Consider the 
two probabilistic automata Mi and Mi of Figure 8-3. Mi contains the transitions of M\ plus 
a transitions that is obtained by combining probabilistically the transitions of M\. For this 
reason there is no simulation from Mi to M\ (the additional transition cannot be simulated). 
On the other hand, M\ and Mi have exactly the same probabilistic executions, and therefore 
we do not see any reason to distinguish them. ■ 

Example 8.3.1 suggests two new relations, which are coarser than strong bisimulation and strong 
simulation, where the only difference is that a transition can be simulated by a probabilistic 
combination of transitions. 

For convenience assume that M\ and Mi do not have common states. A strong probabilistic 
bisimulation between two simple probabilistic automata M\,Mi is an equivalence relation 1Z 
over states(M\) U states^M?) such that 

1. each start state of Mi is related to at least one start state of M2, and vice versa; 

2. for each pair of states si 1Z S2 and each transition si — ► V\ of either Mi or M2, there 
exists a combined transition si — >c ^2 of either Mi or M2 such that V\ =-jz Vi- 

We write Mi ~p M2 whenever acts(M\) = acts^M?) and there is a strong probabilistic bisim- 
ulation between Mi and M^- 

A strong probabilistic simulation between two simple probabilistic automata Mi and M2 is 
a relation 1ZC states(M\) X states(M2) such that 

1. each start state of Mi is related to at least one start state of M2; 

2. for each pair of states si 1Z S2 and each transition si — ► V\ of Mi, there exists a combined 
transition si — >c ^2 of M2 such that V\ ^-jz Vi- 

We write Mi Cgpg M2 whenever acts(M\) = acts^M?) and there is a strong probabilistic 
simulation from Mi to M^- We denote the kernel of strong probabilistic simulation by =sps- 
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It is easy to check that ~p is an equivalence relation, that Cgpg is a preorder relation, and 
that both ~p and Cgpg are preserved by the parallel composition operator. It is easy as well 
to verify that a strong bisimulation is also a strong probabilistic bisimulation and that a strong 
simulation is also a strong probabilistic simulation. 

8.4 Weak Probabilistic Simulations 

The abstraction from internal computation can be obtained in the same way as for ordinary 
automata: a transition of a probabilistic automaton should be simulated by a collection of 
internal and external transitions of another probabilistic automaton. For the formal definition 
we use the weak combined transitions of Chapter 4. 

For convenience assume that Mi and Mi do not have common states. A weak probabilistic 
bisimulation between two simple probabilistic automata Mi and Mi is an equivalence relation 
1Z over states(M\) U states(M~2) such that 

1. each start state of Mi is related to at least one start state of M2, and vice versa; 

2. for each pair of states si 1Z S2 and each transition si — ► V\ of either Mi or M2, there 
exists a weak combined transition S2 =^c ^2 01 either Mi or M^ such that Vi =-ji 7-V 

We write Mi =p M2 whenever ext(Mi) = ext(M2) and there is a weak probabilistic bisimulation 
between Mi and Mi- 

A weak probabilistic simulation between two simple probabilistic automata Mi and M2 is a 
relation 1ZC states(Mi) X states(M~2) such that 

1. each start state of Mi is related to at least one start state of M2; 

2. for each pair of states si 1Z S2 and each transition si — ► Vi of Mi, there exists a weak 
combined transition s^ =^c ^2 of M2 such that Vi Q-ji V^- 

We write Mi Cyypg M2 whenever ext(Mi) = ext(M2) and there is a weak probabilistic simula- 
tion from Mi to Mi- We denote the kernel of weak probabilistic simulation by =wps- 

It is easy to verify that a strong probabilistic bisimulation is also a weak probabilistic 
bisimulation and that a strong probabilistic simulation is also a weak probabilistic simulation. 
However, it is not as easy to verify that =p is an equivalence relation, that Cyypg is a preorder 
relation, and that both =p and Cyypg are preserved by the parallel composition operator. The 
verification of these properties is a simplification of the verification of the same properties for 
the relation of the next section. For this reason we omit the proofs from this section. 

8.5 Probabilistic Forward Simulations 

One of the main results of this chapter is that all the relations presented so far are sound for 
the trace distribution precongruence. However, none of the relations of the previous sections 
allow for one probabilistic operation to be implemented by several probabilistic operations. 
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Figure 8-4: Implementation of a probabilistic transition with several probabilistic transitions. 
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Figure 8-5: A more sophisticated implementation. 

Example 8.5.1 (Weak probabilistic simulations are too coarse) Consider the two prob- 
abilistic automata of Figure 8-4. The probabilistic automaton M2, which chooses internally one 
element out of four with probability 1/4 each, is implemented by the probabilistic automaton 
Mi, which flips two fair coins to make the same choice. However, the first transition of Mi 
cannot be simulated by M2 since the probabilistic choice of M2 is not resolved completely yet 
in Mi. This situation suggests a new preorder relation where a state of Mi can be related 
to a probability distribution over states of M^- The informal idea behind a relation si 1Z V2 
is that si represents an intermediate stage of Mi in reaching the distribution Vi- For exam- 
ple, in Figure 8-4 state si would be related to a uniform distribution V over states S3 and s' 4 
(V = M(s' 3 , S4)), meaning that si is an intermediate stage of Mi in reaching the distribution V . 
It is also possible to create examples where the relationship between s and V does not mean 
simply that s is an intermediate stage of Mi in reaching the distribution V, but rather that 
s is an intermediate stage in reaching a probability distribution that can be reached from V . 
Consider the two probabilistic automata of Figure 8-5. Although not evident at the moment, 
Mi and M2 are in the trace distribution precongruence relation, i.e., Mi C_dc M2. Following 
the same idea as for the example of Figure 8-4, state si is related to U(s' 3 ,s' 4 ). However, si is 
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not an intermediate stage of Mi in reaching U(s' 3 , s 4 ), since si enables a transition labeled with 
an external action /, while in Mi no external action occurs before reaching U(s' 3 ,s' 4 ). Rather, 
from S3 and s 4 there are two transitions labeled with /, and thus the only way to simulate 
the transition si — ► ^(53,54) from ZY(s 3 ,s 4 ) is to perform the two transitions labeled with 
/, which lead to the distribution U(s' 7 , s 8 , s' 9 , s' w ). Now the question is the following: in what 
sense does U(s' 7 , s' 8 , s' 9 , s' w ) represent U{s 3 ,s 4 )1 The first observation is that S3 can be seen as 
an intermediate stage in reaching U(s' 7 , s§), and that S4 can be seen as an intermediate stage in 
reaching U{s'q,s' w ). Thus, S3 is related to U{s 7 ,s 8 ) and S4 is related to U{s'q,s' w ). The second 
observation is that U(s' 7 , s§, s' 9 , s' w ) can be expressed as l/2L((s 7 ,s 8 ) + l/2U(s' 9 , s' w ). Thus, 
U(s' 7 , Sg, s' 9 , s' w ) can be seen as a combination of two probability spaces, each one representing 
an element of U(ss, S4). This recalls the lifting of a relation that we introduced at the beginning 
of this chapter. ■ 

Based on Example 8.5.1, we can move to the formal definition of a probabilistic forward simu- 
lation. A probabilistic forward simulation between two simple probabilistic automata Mi and 
M2 is a relation 1ZC states(M\) X Probs(states(M2)) such that 

1. each start state of Mi is related to at least one Dirac distribution over a start state of 

M 2 ; 

2. for each s 1Z V', if s — ► Vi, then 

(a) lor each s £ il there exists a probability space V s > such that s =^c >v 5 an d 

(b) there exists a probability space V' 2 of Probs(Probs(states(M2))) satisfying V\ ^-jz V2, 

such that Zs'en* P'[s']V a > = Even* P^W- 

We write Mi C_fs M2 whenever ext(M\) = ext(M2) and there is a probabilistic forward simu- 
lation from Mi to M^- 

Example 8.5.2 (A probabilistic forward simulation) The probabilistic forward simula- 
tion for the probabilistic automata Mi and M2 of Figure 8-5 is the following: sq is related 
to ZY(sq); each state s 8 -, i > 7, is related to V(s[); each state s 8 -, 1 < i < 6, is related to 
^( s 2i+n s 2i+2)- ^ 1S an eas Y exercise to check that this relation is a probabilistic forward 
simulation. Observe also that there is no probabilistic forward simulation from M2 to M\. In- 
formally, s 3 cannot be simulated by Mi, since the only candidate state to be related to s' x is si, 
and si does not contain all the information contained in s 3 . The formal way to see that there 
is no probabilistic forward simulation from M2 to Mi is to observe that M2 and Mi are not in 
the trace distribution precongruence relation and then use the fact that probabilistic forward 
simulations are sound for the trace distribution precongruence relation (cf. Section 8.7). In 
M2||Cp it is possible force action left to be scheduled exactly when M2 is in s 3 , and thus it 
is possible to create a correlation between action left and actions a and b; in Mi||Cp such a 
correlation cannot be created since action left must be scheduled before action /. ■ 

It is easy to check that a weak probabilistic simulation is a special case of a probabilistic forward 
simulation where each state of Mi is related to a Dirac distribution. The verification that C_fs 
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is a preorder that is preserved by parallel composition is more complicated. In this section 
we show that C_fs is preserved by parallel composition; the proof that C_fs is a preorder is 
postponed to Section 8.6.4. 

Proposition 8.5.1 C_fs is preserved by the parallel composition operator. 

Proof. Let Mi C_fs M2, and let 1Z be a probabilistic forward simulation from Mi to M 2 . Let 
1Z' be a relation between states(M\) X states(M^) and Probs(states(M 2 ) X states(M^)), defined 
as follows: 

Oi, s 3 ) W V iff P = V 2 ® V(s 3 ) for some V 2 such that 5 X U V 2 . (8.4) 

Condition 1 of the definition of a probabilistic forward simulation is immediate to verify. Con- 
dition 2 for transitions that involve Mi only or M3 only is immediate to verify as well. 

Let (51,53) 1Z' V 2 ® V(s^), and let (51,53) — > V\ ® V3, where si — > Vi, and 53 — > 
V3. From the definition of a probabilistic forward simulation, for each 5 £ £l 2 there exists 
a weak combined transition s 2 =^c V s of M 2 , and there exists a probability space V' 2 of 
Probs(Probs(states(M 2 ))) , such that 



£ p Mv s = £ iflm (8.5) 

and 



sen 2 ven' 2 



V x C K V' 2 . (8.6) 

For each s £ £l 2 , let C? s be a generator for 5 =^c V s . Define a new generator 0' s as follows: 
for each finite execution fragment a of M2HM3 starting in (5, 53), 

1. if O s (a\M 2 ) = (s',V), where (s',V) = ^2iPi(s',ai,Vi), each (s',a,i,Vi) is a transition of 

M2, and a\M 3 = 53, then 

, s (a) = ^2 P i((s , ,s 3 ),a i ,V i ®Vl), 

i 

where 

V[ = T>(ss) if ai ^ a, and V[ = V 3 if a; = a. 

2. if O s (a\M 2 ) = (s',V), where (s',V) = ^2iPi(s',ai,Vi), each (s',a,i,Vi) is a transition of 
M2, a\M 3 = 53053, and 5 3 G O3, then 

<%(«) = !>((*', 4), a.-, ^ ® z>(4)); 

i 

3. if none of the above cases holds, then 0' s (a) = V(8). 



175 



The weak combined transition generated by each 0' s is (s, S3) =^c V S ®V 3 . In fact, an execution 
fragment a of M2HM3 is terminal for 0' s iff a\M 2 is terminal for O s and a\M 3 = s^as' 3 for 
s 3 G O3, and thus Op/ = S7 S X O3. Moreover, for each a G fiev, P a s = P a (^j P3[lstate(a\ M3)]. 
Denote V s ® V3 by Vt StS3 y Then, for each (s,ss) G ^2 X V(ss), we have identified a weak 
combined transition (s, S3) =^c Vt SjS3 \. These are the spaces of Condition 2. a in the definition 
of a probabilistic forward simulation. Note that Vr s S3 \ can be expressed alternatively as 

V(s,s 3 ) = E p s[4] (Vs ® X»(4)) . (8.7) 

Let 

n 3 = e p 3 [4](^®^(^(4))), (s.s) 

4efi 3 

where the pairing of two probability spaces is meant to be their product. For each s 3 G O3, 
since V\ Qn V 2 , V\ ® V(s' 3 ) C K V' 2 ® V(V(s 3 )). Thus, from Lemma 8.2.1, V\ ® V 3 Qn V 2 ^. 
This is enough to show that Condition 2.b of the definition of a probabilistic forward simulation 
is satisfied. 

We are left with J2.seQ2 P^[ s ]V( s ,s 3 ) = J2veQ' P'l 3[V]V, which is shown as follows. From (8.7), 

E P ^]V(s,s 3 ) = E E P ^}P3[4} (Vs ® V(s' 3 )) . (8.9) 

sefi 2 SEO2 s' 3 en 3 

From (8.5), 

E PMv ( s,s 3) = E E nv]P 3 [4] (v ® i>(4)) . (8.10) 

sen 2 s' 3 en 3 ven' 2 

From a simple algebraic manipulation, 

E P MV ( s,s 3 ) = E E Psi4]P2[V]V. (8.11) 

sen 2 s ' 3 eU 3 Ven v^i8,v(v(s' 3 y) 

From (8.8), 

T,P^]V(s,s 3 )= E P 2,3^F- (8-12) 

sefi 2 Ten' 



8.6 The Execution Correspondence Theorem 

The existence of some simulation relation between two probabilistic automata implies that there 
is some strict relation between their probabilistic executions. This relationship is known as the 
execution correspondence lemma for ordinary automata [GSSL94] and is useful in the context 
of liveness. In this section we prove the execution correspondence theorem for probabilistic 
automata; a corollary, which is proved in Section 8.7, is that the existence of a probabilistic 
forward simulation is sound for the trace distribution precongruence. 
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Figure 8-6: Fringes. 



8.6.1 Fringes 



Let if be a probabilistic execution of a probabilistic automaton M. Define the extended states 
of H, denoted by extstates(H), to be states(H) U {q8 \ q £ states ( H ), PH[C q $] > 0}. A fringe 
of if is a discrete probability space V of Probs(extstates(H)) such that for each state q of if, 



g'Gfi|g<g' 
Two fringes Pi and P2 are in the < relation iff for each state g of if , 



(8.13) 



(8.14) 



Informally, a fringe is a line that cuts a probabilistic execution in two parts (see Figure 8-6). A 
fringe is smaller than another one if the first fringe cuts the probabilistic execution earlier than 
the second fringe. Figure 8-6 shows three fringes i\,i2 and F3, where i\ < Fi < F3. 

A fringe of particular interest is the fringe that cuts a probabilistic execution fragment at 
some depth i. Let fringe(H,i) denote the fringe of H where S7 = {q £ extstates(H) \ \q\ = 
i} U {qS £ extstates(H) \ \q\ < i}, and for each q £ 0, P[q] = PH[C q ]. 

8.6.2 Execution Correspondence Structure 

Let 1Z be a probabilistic forward simulation from Mi to M2. An execution correspondence 
structure via 1Z is a tuple (ifi,if2, to, 5), where ifi is a probabilistic execution of Mi, #2 is a 
probabilistic execution of M2, mis a mapping from natural numbers to fringes of M2, and S 
is a mapping from natural numbers to probability distributions of Probs(Probs(states(H2))) , 
such that 

1. For each i, m(i) < m(i + 1); 

2. For each state q 2 of if 2 , lim 8 ^oo E g efi 8 |92<g P do\ = p H[C q ]; 

3. Let qi 1Z V iff for each g £ 0, trace(q) = trace(qi), and either 

(a) gi does not end in 8, each state of i7 does not end in 8, and lstate(q\) 1Z Istate(V), 
or 

(b) qi and each state of S7 end in <*) and lstate(8- strip(qi)) 1Z lstate(8- strip(V)) . 
Then, for each i > 0, m(i) = Y.ven s(l) p S(i)[P\P-, and fringe(H u i) Q n S(i). 
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Figure 8-7: Execution Correspondence Structures: the role of Condition 2. 

4. Let, for each i > 0, each q\ G fringe(H\,i), and each ^2 £ states(H 2 ), Wi(q\,q2) = 
J2v w i( ( l^^)^'[ ( l'i\- If ^'(ft?^) = f° r eacn prefix or extension g 2 of q^t then, for each 
extension q[ of gi such that q[ G fringe(H\,i + 1) and each prefix or extension q' 2 of q^ 
W i+ i( 9 i,^) = 0. 

Informally, an execution correspondence structure is an object that shows how a probabilistic 
execution Hi of Mi is represented by a probabilistic execution H2 of M2 via 7^.. i?2 is said to 
be the probabilistic execution fragment that corresponds to Hi. Conditions 1 and 3 state that 
each fringe fringe(H\,i) is represented by the fringe m(i) in H2, and Condition 2 states that 
at the limit each state of H2 represents some part of Hi. Figure 8-7 gives an example of an 
execution correspondence structure (left) and of a structure that fails to satisfy Condition 2 
since state q is not captured (right). Condition 4 enforces the correspondence between Hi and 
H2. Informally, it states that if two states qi and ^2 of H\ and H2, respectively, are connected 
through the i fringes, then for each j < i there are two prefixes q[ and q' 2 of q\ and ^7 
respectively, that are connected through the j th fringes. This condition allows us to derive a 
correspondence structure between the execution fragments of Mi and M2 that denote the states 
of H\ and H.^- We do not use Condition 4 to prove any of the results that we present in this 
thesis; however, this condition is necessary to prove the results that Segala and Lynch present 
in [SL94]. 

If 1Z is a weak probabilistic simulation, then an execution correspondence structure is a 
triplet (Hi,H2,m): Condition 3 becomes fringe(H\,i) C^ m(i), where q\ 1Z q2 iff trace(qi) = 
trace(q2) and either q\ and ^2 en d in 8 and 8- strip(lstate(qi)) 1Z 8- strip(lstate(q2)) , or Istate(qi) 1Z 
lstate(q2); Wi(qi,q2) becomes Wi(qi,q2), and Condition 4 says that for each i > 0, given 
^i G fringe(Hi,i) and ^2 & states^Hz), if Wi(qi,q' 2 ) = for each prefix or extension q' 2 of q2, 
then, for each extension q[ of qi such that q[ G fringe(H\,i + 1), and each prefix or extension 
q' 2 of q 2 , w l+ i(q[,q' 2 ) = 0. 
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If 1Z is a strong probabilistic simulation, then an execution correspondence structure is a pair 
(Si, S2): Conditions 1 and 2 are removed; Condition 3 becomes fringe(Si, i) C^ fringe(S2, i) 
where qi 1Z qi iff itrace(qi) = itrace(q2) and either qi and g 2 end in S and S- strip(lstate(qi)) 1Z 
S-strip(lstate(q2)) , or Istate(qi) 1Z /state(g 2 ); Condition 4 says that for each i > 0, given qi G 
fringe(Si, i) and qi G fringe(S2, i), if Wi(q\, ^2) = 0, then, for each extension q[ of qi such that 
q[ G fringe(Si, i+1) and each extension g 2 °f <?2 such that q' 2 G fringe(S2, i+1), w 8 _|_i(g 1 , g 2 ) = 0. 

8.6.3 The Main Theorem 

Theorem 8.6.1 Let Mi C_fs M2 via the probabilistic forward simulation 1Z, and let Si be a 
probabilistic execution of Mi. Then there exists a probabilistic execution S 2 of M 2 , a map- 
ping m from natural numbers to fringes of M 2 , and a mapping S from natural numbers to 
probability distributions of Probs(Probs(states(S 2 ))), such that (Si,S 2 ,m,S) is an execution 
correspondence structure via 1Z. 

Proof. Let qi be a state of Si, and let V2 be a distribution over potential states of S 2 such 
that qi C.-JI V2 according to the definition given in the definition of an execution correspondence 
structure. Denote by Vfj the probability space such that tr^ 1 = J2treQ qi ^H [^ r ](li^ ^ r )- Let 

tr\ G Qff , and let Vt ri be the probability space reached in qi ~ tr\. 

Since 1Z is a probabilistic forward simulation, then for each state q 2 of S7 2 there exists a 
weak transition tr qi -p 2iriq2 of S2 with action a \ ext(M2), leading to a distribution over states 
V qi -p 2 tr 1 q 2 , such that there exists a probability distribution over probability distributions of 
potential states of S2, denoted by V q -p tr , satisfying 

E p iv 2 i ri [P]V= J2 P2Vh]V qi v 2 tr iq2 (8.15) 



v ^ S n v 2 t ri " 2Gfi2 



and 



V tri Qn V s qiV2tri (8.16) 

via a weight function w qi -p 2 t Tl ■ Denote the probability space J2q 2 eQ 2 Si\ ( li\P qi T 2 ir iq2 by V qi -p 2 t ri , 
i.e., 

V qi V 2 ir 1 = 2-^ P2[q2\P qi V 2 tr iq2 - (8-17) 

Denote the generator of each weak transition tr qi -p 2iriq2 by O qi -p 2 t Tiq2 (cf. Section 4.2.7). For the 
sake of this proof, we change the notation for the generators of the transitions of a probabilistic 
execution. Thus, for each q' 2 such that g 2 < Q^ ^qiT 2 tr iq2 ((i2) stands for O qi -p 2 t riq2 (q 2 ]' q2) , and 

p° n V 2 tr iq2 standg for p O qi V 2 tr iq2 _ 
1 2 1 2 \12 

For each state qi and each probability distribution over states V2, let S qi = V(q\8), 6-p 2 = 

J2 q2 en 2 P 2il^q2^v 2 = V (h 2 ), and v) Sqi v 2 be a weight function such that w 8qiV2 (qib,V2) = 1. 
Note that, if for each q2 £ ^2, trace(qi) = traced), then 

S qi Ek 4 2 (8-18) 
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via ws qi -p 2 . Moreover, 

^ = E p ^ PF- ( 8 - 19 ) 

Let si be the start state of H\, and s 2 be a start state of M 2 that is related to si. We know 
that s 2 exists since 1Z is a probabilistic forward simulation. Let Active be the smallest set such 
that 

1. (si,X>(s 2 )) G Active; 

2. if (gi,P 2 ) G 4cft've, tr x G fi^, and (gi,P 2 ) G ^fn X ^^ 2in , then (gi,P 2 ) G ^cft've; 

3. if (qi,V 2 ) G Active, Pfj^S] > 0, then (giM| 2 ) G ^cfwe. 

Observe that for each pair (gi,P 2 ) G Active, q\ 1Z V2 (simple inductive argument). For each q\ 
such that there exists some V2 with (gi,P 2 ) G Active, each tr\ G fi^ , and each g 2 G fi 2 , let 
active(qi,V2, tr\, g 2 ) be the set of states that are active in O qi -p 2 t riq2 , and let reach(qi,V2, tr\, g 2 ) 
be the set of states that are reachable in O qi -p 2 t riq2 . Let active denote the union of the sets 
reach(qi,V2,tri,q2) where (gi,P 2 ) G Active, tr\ G Q 9 jj , and g 2 G fi 2 . For each i < 0, let 
Active(i) be the set of pairs (gi,P 2 ) G Active such that either |gi| = i or |gi| < i and gi ends 
in 8. For each pair (gi,P 2 ) of Active such that gi does not end in 8, let 

^1 = E ^[friFtn+^JW,! (8.20) 

friend 

be the probability space reached in Hi with the transition enabled from q\, 

V qi v 2 = E PH\[tri]V qi -p 2tri + Ph\[8]h 2 (8-21) 

be the probability space that is reached in the corresponding transition of V2, 

Kv 2 = E ^i]^n+W^ (8-22) 

be the probability space of probability spaces that corresponds to V qi , and for each q^V^, 



w qi vM^2) = E PKitriHMtrM^ + P&WvsnvMM) (8-23) 

be the corresponding weight function. From Lemma 8.2.1, 

V<n Ek V s qiV2 (8.24) 

via the weight function w qi -p 2 . 

For each pair (gi,P 2 ) of Active such that gi ends in 8, let 

7> 31 = V( qi ), V quV2 = V 2 , V s quV2 = V(V 2 ), znd w qiV2 ( qi ,V 2 ) = 1. (8.25) 

It is immediate to observe that Equation (8.24) holds also in this case. 
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Define m(i),S(i) and W{ inductively as follows. 

m(0) = V(s 2 ), S(0) = V(m(0)), w o ( Sl ,m(0)) = 1, (8.26) 

m(i+l) = E Wi(qi,V 2 )V qiV *, (8-27) 

^ + i) = E MqiMKv*' ( 8 - 28 ) 

w,-+i(?i,^) = X) ^(gi,P 2 )^^ 2 (gi,^2)- (8-29) 

To show that Equations (8.27), (8. 28), and (8.29) are well defined, we show by induction that 
for each i > 0, J2( q v )eActive(i) w i( ( l^^'i) = 1- The base case is a direct consequence of (8.26) 
and the definition of Active(O). For the inductive step, 

E w l+1 (q 1 ,V 2 ) 

(gi,V 2 )eActive(i+l) 

E E Mll^^Wq^ill,^) 

(qi,V 2 )EActive(i+l) (q[,T! 2 )eActive(i) 

E Wi(q[,V^) 

(q[,V 2 )eActive(i) 
= 1, 

where the first step follows from Equation (8.29), the second step follows from the fact that 
w q i -pi is a weight function that is non zero only in pairs of Active(i + 1), and the third step 
follows from induction. Let 

W qi v 2 tr ig M) = w( qi ,V 2 )P q H \[t ri ]P 2 [q 2 ]P° nV2tr ^ . (8.30) 

Consider a state q 2 of active. Then the transition enabled from q 2 is 

E E E (8-3i) 

{q'^V'^^Active t ri £Q H l q 2 e^i' 2 \q2&active(q' 1 ,V 2 ,tr 1 ,q 2 ) 



P ^ lP2t r 1 4(92)[ acfe ( M 2)]W / g ;^ i n^(^)/W / (g 2 ) [O q , iV2triq , 2 (q 2 ) \ acts(M 2 ] 
where W{s 2 ) = 1, and for each q 2 ^ s 2 , 



W(q 2 ) = E E E W q[v , triq ,(q 2 ). (8.32) 

(q[,V 2 )eActive ir gj) fl l q' 2 e^i' 2 \q' 2 ^q2,q2&'reach(q' 1 ,V 2 ,tr 1 ,q' 2 ) 
q 'l 

It is easy to verify that Expression (8.31) denotes a valid transition of a probabilistic execution 
fragment of M since it is the combination of legal transitions of a probabilistic execution 
fragment of M. The fact that the projection of a legal transition of a probabilistic execution 
fragment of M onto acts(M) is still a legal transition of a probabilistic execution fragment of 
M follows from the fact that M is a simple probabilistic automaton. 
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Informally, the set active is used to identify all the states of H 2 . The transition enabled from 
each one of those states, say q 2 , is due to several states of Hi, and each state of Hi influences 
the transition enabled from a specific state of Hi with a different probability. Such a probability 
depends on how much a state of Hi represents a state of Hi, on the probability of the transition 
of Mi that has to be matched, on the probability of reaching a specific state q 2 of Hi during 
the matching operation, on the probability of reaching q 2 from q' 2 , and on the probability of 
departing from q 2 . These conditions are captured by Pq , ( q2 )[acts(M 2 )]W q i-pi triq i (q 2 )- 

These weights must be normalized with respect to the probability of reaching q 2 , which is 
expressed by W(q 2 ). The condition q' 2 7^ q 2 in the third sum of (8.32) is justified by the fact 
W(q 2 ) is the probability of reaching q 2 . 

This completes the definition of Hi, m(i), S(i), and the w^s. We need to show that 
(Hi, H2, w, S) is an execution correspondence structure via 1Z. Thus, we need to show the 
following properties. 

1. For each i, m(i) is a fringe of Hi', 

2. For each i, m(i) < m(i + 1); 

3. For each state q of H 2 , lim^^ E g 'efi 8 |g<g' P M'\ = p H[C q ]; 

4. For each i, m(i) = J2ves( t ) p s{i)[P]V; 

5. For each i, fringe(H\,i) C^ S(i) via W{. 

6. For each i, each qi £ fringe(Hi,i), and each q 2 G states(H2), if Wi(qi,q' 2 ) = for each 
prefix or extension q 2 of q 2 , then, for each extension q[ of qi such that q[ £ fringe(Hi, i + 1) 
and each prefix or extension q 2 of q 2 , T / Fj_|_i(g 1 , q 2 ) = 0. 

We show each item separately. 

1. For each i, m(i) is a fringe of H 2 . 

By construction m(i) is a probability distribution. Thus, we need to show only that for 
each state q 2 of H 2 , 

£ Pm^] < P H 2 [C q2 ] (8.33) 

<l'2^m(i)\<l2<q' 2 

First we show that for each q 2 £ states(H 2 ), 

W(q 2 ) = P H2 [C q2 }- (8.34) 

then we show that for each q 2 £ states(H 2 ), 

£ Pm(iM] < W(q 2 ). (8.35) 

<l2^m(i)\<l2<q 2 

The proof of (8.34) is by induction on the length of q 2 . If q 2 = s 2 , then (8.34) holds by 
definition. Otherwise, let q 2 be q 2 without its last action and state, i.e., q 2 = q 2 as for 
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some action a and some state s. Then, from the definition of the probability of a cone, 
induction, Equation (8.31) and an algebraic simplification, 



p H2 [c q2 ]= E E E 

{q^V 2) €. Active q[ q 2 ^^2 \q2^^ctive(q' 1 ,V! 2 ,tri,q' 2 ) 

W q[vitrW2 (h)Po q , V2triq , 2 (g 2 )[q2}. (8.36) 



From Equation (8.30) and the definition of P q2 1 2 2 (cf. Section 4.2.7), we obtain 

p H2 [c q2 ]= E E E 

(q[,V^)eActive q[ q^e^l^eactiveiq^^^tr!^) 

^i^P^tr^PUwt^' 2 ^' 2 ■ (8-37) 

Observe that q' 2 G fi 2 anc ^ #2 & aciife(g^ , T^^? ^ r i ? ^2) iff Q2 ^ ^2> #2 7^ ?2 5 an d g 2 £ 
reac/i(g 1 ,7 7 2? ^ r i? ^2)- Thus, from Equation (8.31), 

Pn 2 [C q2 ]= E E E ^^(?2). (8-38) 

(q[,V 2 )EActive q[ q 2 EQ 2 \q 2 ^q 2 ,q 2 Ereach(q' 1 ,V 2 ,tr 1 ,q 2 ) 

At this point Equation (8.32) is sufficient to conclude the validity of Equation (8.34). 

The proof of Equation (8.35) is also by induction. If i = 0, then the result follows directly 
from the fact that a fringe is a probability distribution. Otherwise, let N(q\) be true iff 
qi does not end in 8. Then, from Equation (8.27), 

E Pm(i+l)W2\ (8-39) 

q 2 e^m( t + 1 )\q 2 <q 2 

can be rewritten into 

E E w t (q 1 ,V 2 )P qi v 2 [q' 2 }. (8.40) 

q 2 eQ m ( t + 1 )\q 2 <q 2 (qi,V 2 )eActive(i) 

From the definition of V qit -p 2 (Equations (8.21) and (8.25)) and the definition of V qi -p 2 t Tl 
(Equation (8.17)), Expression (8.40) can be rewritten into 

E E E E (8-4i) 

q' 2 en m(t + 1) \q 2 <q' 2 ( qi ,V 2 )eActive(i),N ( qi ) f ri Gfi^ q 2 & 2 

^(gi,^2)P^ 1 [^ 1 ]P 2 [g 2 ']P gi ^ 2irig »[g 2 ] 
+ E E ".-(fc.W&ffliM 

q 2 Se^ m ( t + l)\l2<q 2 (qi,V 2 )eActive(i),N(q 1 ) 

+ E E m(q 1 8,V 2 )P 2 [q' 2 8}. 

q 2 8e£lm( t + 1 )\q 2 <q 2 (qiS,~P 2 )eActive(i) 
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By exchanging sums in Expression (8.41), we obtain 

E E E E (8-42) 

(qi,V2)eActive(i),N( qi ) ir-iGfig q^eU 2 q' 2 £Si m{i + 1) \q2<q' 2 

^■(?i,^2)i^ l 1 [<r 1 ]P 2 [^]P 3l7 , 2iri3 „[^] 

+ E E w l ( qi ,V 2 )Pg[S]P 2 [q' 2 ] 

(qi,'P2)eActive(i),N(q 1 ) q 2 Se^ m ( t + l)\q2<q 2 

+ E E ^(ft^)^^], 

(qi6,V 2 )eActive(i) q 2 Se^ m ( t + l)\q2<q 2 

where the first summand comes from the first summand of (8.22), the second summand 
comes from the second summand of (8.22), and the third summand comes from (8.25). 
Consider the first summand of Expression (8.42), and partition the states q 2 °f ^2 into 
those that include q 2 (q 2 < q 2 ) and those that do not. In the first case, since from (8.27), 
(8.21), and (8.17), Q qi -p 2 t riq " ^ &m(i+i)i an( i si nce each element q 2 of Q qi -p 2 t riq " satisfies 
12 < q' 2 , 

E Pq^tr^W,] = i; (8-43) 

q 2 eQ m (i+i)\q2<q' 2 

in the second case the same sum gives P q2 2 . Consider the second summand of 

Expression (8.42), and observe that, from (8.27), (8.21), and the definition of 8-p 2 , q 2 8 G 
& m (i+i), ([2 < ?2' an( i -^M*?^] > iff q 2 G O2 , q 2 < q' 2 , and i^f^] > 0- Finally, consider 
the third summand of Expression (8.42), and observe that all the states of £l 2 en d with 8, 
and, from (8.27) and (8.21), q 2 8 G &m(i+i)i ( h < Q 2 i an d P 2 [q' 2 8] > iff g^ G O25 <?2 < #2 ^5 
P2[q' 2 fi] > 0. By combining the observations above, Expression (8.42) can be rewritten 
into 

J2 E MQi^Pgitri] (8.44) 

(qi,V2)eActive(i),N( qi ) tnen'g 

( E PM+ E ^nt 91 ^ 19 ") 

\g 2 'efi 2 |g2<g 2 ' q"e^ 2 \q 2 <q 2 J 

+ E E f.-fo.w&ffliM 

+ E E ^-(ffi^^)^^']- 

(qi8,V 2 )eActive(i)q^eU 2 \q 2 <q' 2 ' 

By regrouping expressions and simplifying, we obtain 

E E E w^MP^tr^iq^P^^ (8.45) 

(qi,v 2 )eActive(i),N( qi ) tnen'g q"en 2 \q 2 <q 2 ' 

+ E E ^(ft,^)^"]- 

(gi,^2)e^c<i-!;e(i)g2'e fi 2k2<g 2 ' 
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Finally, from Equation (8.30), Expression (8.45) can be rewritten into 

E E E W qiV2triq ,,(q 2 ) (8.46) 

(qi,v 2 )eActive(i),N( qi ) tr 1 en'g q 2 en 2 \ q2 < q i! 

+ E E ^(ffi,^)^^']- 

(gi, V 2 )eActive(i) q 2 e^ 2 \q 2 <q 2 

We now analyze the second summand of Expression (8.46), and we show by induction on 
i that it is if i = and q 2 ^ s 2 , it is 1 if i = and q 2 = s 2 , an d it is 

E E E E W> 2 *r ig »(?2) (8.47) 

3<i (qi,V 2 )e Active^) i rie S)« g^'efi 2 |g"<92 

otherwise. For i = the result is trivial. Otherwise, from Equation (8.29), 

Y, E w l+1 (q u V 2 )P 2 [q'i\ (8.48) 

(qi,V 2 )eActlve(l + l)q^eU 2 \q 2 <q^ 

can be rewritten into 

E E E w l (q l 1 ,V l 2 )w q , n (q 1 ,V 2 )P 2 [q' 2 1 }. (8.49) 

(gi,^2)e4c^e( 8 +i)(g;,^)e4c^e( 8 )g"e"2k2<g2' 

From the definition of w q i-pi (Equations (8.23) and (8.25)), Expression (8.49) can be 
rewritten into 

E E E E (8-50) 

(qi,V 2 )eActive(i+l) (g' 1 ,T^eActive(i),N(g' 1 ) , g( g^Gfi 2 \q 2 <q" 

^i^^J^Kw^i,^))^'] 
+ E E E 

(gi5,^2)e^c^e( 8 + l) (gJ,^)G^c^e( 8 ),iV(g;) q'^^ 2 \q 2 <q'^ 

wH,V' 2 )P q ^ [S}w 5q , v ,( qi 6, V 2 )P 2 [q' 2 '} 

+ E E wi&WPtWA- 

(q'^^eActiveii) q^en' 2 \q 2 <q^ 

Observe that in the first summand of (8.50) 

E E W q\V' 2 tr\ (ft, Vl)Pl\<&\ 

(qi,V 2 )eActive(i + l)q^eU 2 \q 2 <q^ 

E E PqUir^m^] 

T2\Bgi,(gi,T 2 )eActive(i+l)g^eQ2\g2<g2 

= E E P q[V^r' iq !J'W 2 ], 

q' 2 "eV> 2 q>'eV q , , tr ,\q 2 <q>> 
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where the first step follows from the fact that Wq'-p'tr 1 q 1 " is a weight function, and the 
second step follows from (8.17), (8.15) and the fact that Q, i-pi ir i is the set of probability 
space V2 such that there is a state q\ where (gi,^) & Active(i + 1) (cf. the definition 
of Active and observe that \q\\ = i + 1). For the second summand of (8.50), observe 

0' 
that for each pair (q[S, V2) of Active(i + 1), if P^ [S] > 0, then there is exactly one pair 

{qi,V 2 ) of Active(i) such that Wg q i-pi(q[8,V2) > 0. In particular, q\ = q[, V2 = 8-p> , and 

q' 

w Sq i-pi(q' 1 S,V2) = 1- Conversely, for each pair {q'nV' 2 ) of Active(i) such that P^ [6] > 0, 
the pair (q[S, V2) is in Active(i-\-l) and wg q i-pi(q[8, V2) = 1- Thus, the term w$ q i-pt(q l 1 6,V2) 
and the sum J2( q 'sv )eActive(i+i) can ^ e remove d from the second summand of (8.50). 
Thus, by applying the observations above to (8.50), we obtain 

E E E E ( 8 - 51 ) 

(8i,^)€Actit;e(t),JV(gi) oj q 2 " € Q' 2 q'± € Q , „ , tr , ,,,, 1 92 < <tf 

+ E E <^2)<[<^2'] 

+ E E ^i^)^"]- 

Consider the first summand of Expression (8.51). If q2 < q"', then 



E Pqiv'trW'WA = 1; (8-52) 

If q"' < Q2, then 



92'e" q j^trjq»'l92<g" 



E PqiV^q^} = P^^' ■ (8-53) 

92'e" q j^tr; q ^"l92<g" 
Thus, from Equations (8.52) and (8.53), Expression (8.51) can be rewritten into 

E E Mli^PiMK] (8-54) 

(gi,^)€Actit;e(t),/V(gi) gj 

f E *M+ E ^^^ 



/r i"] 



E E Mi'i^PnMmk 

J2 E <3iWM- 

(q'^^^eActiveii) q'^' en' 2 \q 2 <q' 2 " 



186 



By regrouping the subexpressions in (8.54), we obtain 



E E E M&Vti^itr'iWMW 12 l9a (8-55) 

(q'^V^eActive^NCl'i) f , e0 < 1 2 ^' 2 W 2 «l2 

+ E E wU^)pm']- 

(q^^^EActiveii) q' 2 " E£l 2 \q 2 <q' 2 " 

From Equation (8.30), Expression (8.55) can be rewritten into 

E E E W q[v , tr , q ,,,( q2 ) (8.56) 

+ E E wU^)pm']- 

(q[,V! 2 )EActive(i) q'^' EU' 2 \q 2 <q^ 

The induction hypothesis is now sufficient to conciude the vaiidity of (8.47). From an 
aiternative characterization of the set {g" £ ^2 | Q 2 < Q2} in Expressions (8.47) and (8.45), 
and by combining (8.45) and (8.47), we obtain 

E Pm(i+l)W2] (8-57) 

q'2E£i m (i+i)\q2<q' 2 

= E E E E w qiV2triq ,,(q 2 ). 

3<i (gi,V 2 )eActive(j) ir ie S)« q^En 2 \q' 2 ^q 2 ,q^Ereach(q 1 ,T 2 ,tr 1 ,q 2 ) 

Observe that the right expression of (8.57) contains a subset of the terms of the right 
expression of Equation (8.32). This is enough to conclude the validity of (8.35). 

2. For each i, m(i) < m(i + 1). 

This result follows directly from Equation (8.57). In fact, for each state q 2 of H 2 , Ex- 
pression (8.57) for m(i + 1) contains a subset of the terms of the Expression (8.57) for 
m(i). 

3. For each state q of H 2 , lim;-^ E g 'efi 8 |g<g' P M'\ = Pn[C q ]. 

This result follows directly from Expression (8.57). In fact, as i — ► 00, the right expression 
of (8.57) converges to the right expression of (8.32). 

4. For each i, m(i) = J2vES(i) p S(i)[P]V . 

For i = the result is trivial. For i > 0, from Equation (8.27), m(i + 1) is rewritten into. 

E Mqi,V 2 )V qi T 2 . (8.58) 

(91 ,V 2 )EActive(i) 

From Equation (8.21), Expression (8.58) can be rewritten into 



Z^ 



Wi( qi ,V 2 ) I E PH\[tri]V qi -p 2tr2 + P q H \[S]h 2 I • (8.59) 



(qi,V 2 )eActive(i) \ir16fi* 
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From Equation (8.17) applied to V qi -p 2 tr 2 an( i Equations (8.15) and (8.19) applied to 
Pjj [6]6-p 2 , Expression (8.59) can be rewritten into 



w % (qi^ 2 )\ E Ph^I E P q S 1 V 2 ir 1 [nV\+ (8.60) 



y^ „„.<„. t>.\ l v^ p9i 



(qi,V 2 )eActive(i) \tr-,eQ\ V^nV^r, 



Pm E PSvJPW 

ven Sv2 
From Equation (8.22), Expression (8.60) can be rewritten into 



E m(qi,V 2 )\ E P^V 2 [P]P\- (8-61) 

Finally, from Equation (8.28), Expression (8.61) can be rewritten into 

E Ps(i+i)[V]V, (8.62) 

which is what we needed to show. 

5. For each i, fringe(Hi,i) C^ S(i) via W{. 

For i = the result is trivial. By applying the definitions of afringe and of fringe(Hi, i + 1), 

fringe(H 1 ,i+ 1) 

E PHAc qi ]v qi 

qiEstates(H2)\\q2\=ioT q 2 = q' 2 S,\q 2 \<i 

E Wi{q 1 ,V 2 )V qi . 

(qi,V2){zActive(i) 

From (8.28), 

^+1)= E M<li,V2)V s qiV2 . 

(qi,V2){zActive(i) 

Since for each pair (^i,^) of Active(i), V qi Qn V^-p via w qi ^ 2 i from Lemma 8.2.1, 

E M<n,v 2 )V qi c K E Mqi,v 2 )V^ V2 

(<?i ,V2)£z Active(i) (qi ,V2)€.Active(i) 



via Y.( qi ,V 2 )eAct l ve({) W i(<lliP2) W qiV 2 , which is W i+1 . 
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6. For each i, each q\ £ fringe(Hi,i), and each q 2 £ states(H 2 ), if Wi(qi,q 2 ) = for each 
prefix or extension q 2 of q 2 , then, for each extension q[ of q\ such that g x £ fringe(Hi, i + 1) 
and each prefix or extension g 2 °f ?2 5 JFj+i^i^) = 0- 

Suppose by contradiction that there is an extension q[ of q\ such that q[ £ fringe(Hi, i + 1) 
and a prefix or extension q 2 of g 2 such that WiJ r i(q' 1 ,q 2 ) > 0. From the definition of W{ 
and Equation (8.29), 

W t+1 ( qi ,q' 2 ) = J2 E ^■(?i,'P2)^- 1 ,7' 2 (?i,'P)i , [?2]- (8-63) 

V (qi,T 2 )eActive(i) 

Since IF^gi, q 2 ) > 0, then there is at least one probability space V and one pair (gi, V2) £ 
Active(i) such that Wi(qi,V2) > 0, Wq 1) -p 2 (qi,V) > 0, and P[g 2 ] > 0. Then there is at 
least one prefix g 2 of g 2 such that i^f?"] > 0> which means that IF^gi, q 2 ) > 0. However, 
this is a contradiction since g 2 is either a prefix or a suffix of g 2 . 

The execution correspondence theorem can be stated and proved similarly for weak and strong 
probabilistic simulations. The proofs are simpler than the proof presented in this section, and 
thus we omit them from this thesis. 

8.6.4 Transitivity of Probabilistic Forward Simulations 

Now we have enough machinery to prove that probabilistic forward simulations are transitive, 
i.e., if Mi C_fs M 2 and M 2 C_fs M3, then Mi C_fs M3. We start by proving a lemma. 

Lemma 8.6.2 Let (Hi, H 2 , m, S) be an execution correspondence structure via the probabilistic 
forward simulation 1Z, and suppose that Hi represents a weak combined transition s =^c V\. 
Then H 2 represents a weak combined transition s' =>c ^2 and there is a probability space V 2 
such that 

1. ViQn V 2 and 

2- V 2 = Evens Pi[V]V. 

Proof. Let W{ be the weight functions for fringe(Hi,i) C^ S(i). Let V[ be 8-strip(T ) H 1 )i V 2 
be b-strip(Vu 2 ), and let 

Ks = E E w H+ i(aS,V)V. (8.64) 

a8eV Hl V\w\ aHl (aS,V)>0 

For each a 8 £ 0^ and each V £ Probs(extstates(H 2 )), let w(a8,V) = w\ a \ + i(a8,V). 

We show that w is a weight function from V[ to V' 2 s and that V' 2 s is well defined. This im- 
plies that V[ C^ V 2 5. Then we show that for each element a8 of 0# 2 , J2veW -^2 sP 3 ]-^ ^] = 
Ph 2 [Ccxs]- Since all the elements of the probability spaces of S7 2 s end with 8, we obtain that 
V 2 is well defined and that V 2 = J2veQ' -^2 s\PV' '• Then the lemma is proved by defining Vi 
to be lstate(V[), V2 to be lstate(V 2 ), and Vi,s to be lstate(V 2 s ). 

To show that wis a weight function we have to verify the three conditions of the definition 
of a weight function. If w(a8,V) > 0, then, from the definition of w, w\ a \ + i(a8,V) > 0. 
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Since W|„| +1 is a weight function, then a 8 1Z V. Let V G £l' 2 s- Then J2aSeOR w ( a ^P) = 
J2aSeU w \a\+i( a ^P)^ which is P 2 S [V] by definition of V 2 5- Consider now an element a8 of 
Oijj. Then, J2veW w(a8,V) = J2veW w \a\+i( a ^P)- Since W|„| +1 is a weight function, then 
the sum above gives Pjj 1 [C a $] = P[[a8]. To show that V' 2 5 is well defined we need to show that 
J2 a Sen Hl ^V\w H+1 ( a s,r)>o w \a\+i( a ^' P ) = 1 - This follows immediately from the fact that w is a 
weight function and that, since Hi represents a weak combined transition, J2 a SeU P{[a8] = 1. 
We are left to show that for each element a 8 of 0# 2 , J2veQ' ^2 sl^l-f I a ^l = Ph 2 [Ccxs]- 
Observe that for each element a 8 of 0^ , if i < \a\ then Wi(a8, V) is undefined for each V, and 
if i > |a|, then for each j > i and each V, Wi(a8,V) is defined iff Wj(a8,V) is defined, and if 
Wi(a8,V) is defined then Wi(a8,V) = Wj(a8,V). Thus, if we extend each W{ by setting it to 
whenever it is not defined, then, for each a8 £ fi.ff 2 , 

E p 2,sinp[^]= E ( lim E <«<*>, ?>) km]- (s.es) 

■Pefi 2S . v ^ n ' 2 ,s V «5efi ffl / 

Since for each i, W{ is a weight function, and since from the definition of V 2 5 each element V 
for which Wi(a8,V) > is in S7 2 s , then we derive 

E ^2,5[m«*]= E (^P s{i) [V])p[aS]. (8.66) 



7>en' s 7>en' s 



l^OO 



By exchanging the limit with the sum and by using Condition 3 of the definition of an execution 
correspondence structure, the equation above can be rewritten into 

E p 2 s[P]P[aS] = lim m(i)[aS], (8.67) 

which gives the desired result after using Condition 2 of the definition of an execution corre- 
spondence structure. ■ 

Proposition 8.6.3 Probabilistic forward simulations are transitive. 

Proof. Let 1Z\ be a probabilistic forward simulation from Mi to M 2 , and let 1Z 2 be a proba- 
bilistic forward simulation from M 2 to M3. Define 1Z so that si 1Z V3 iff there is a probability 
space P27 an d a probability space V§, such that 

1. si ^1 7>2, 

2. P 2 C K2 Vl, and 

3- ^3 = Evens Pi[-P]V. 

We need to show that 1Z is a probabilistic forward simulation from Mi to M3. For this purpose, 
let si 1Z V3, an( i l e t ^2 an d P3 satisfy the three conditions above. Let si — —> V\. Let M 2 
be obtained from M 2 by introducing a new state S2 and by adding a transition S2 — ► P27 
where r is an internal action; similarly, let M3 be obtained from M3 by introducing a new state 
s 3 and by adding a transition s' 3 — > V3, where r is an internal action. Let TZ[ be obtained 
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from 1Z\ by adding the pair (si,X>(s 2 )), and let 1Z' 2 be obtained from 1Z 2 by adding the pair 
(s 2 ,X , (sg)). Observe that TZ[ is a probabilistic forward simulation from Mi to M 2 and that 1Z' 2 
is a probabilistic forward simulation from M 2 to M3. 

We want to find two probability spaces V 3 and V' 3 s such that S3 =^c V3, ^1 Ek ^3 57 
and P 3 = J2veW -^3 sffiV^- From the definition of a weak transition, this is sufficient to show 

that for each state s of V3 there is a weak combined transition s =^c V s of M3 such that 

n = e sG q 3 p 3H^ s . 

Since VJ X is a probabilistic forward simulation, there is a weak combined transition s 2 =^c 
V 2 of M2 and a probability space V 2 s such that 

V' 2 = E P 2,slV]V and V[Q ni V^ s . (8.68) 

Let i?2 be the probabilistic execution fragment of M 2 that represents the weak combined tran- 
sition s 2 =^c V 2 . Then, by definition of H 2 , V 2 = lstate(8-strip(VH 2 )) ( c f- Section 4.2.7). 

From the Execution Correspondence Theorem there is an execution correspondence struc- 
ture (H 2 , i?3, to, S), where H3 is a probabilistic execution fragment of M 3 that starts from s' 3 . 
From Lemma 8.6.2, H3 represents a weak combined transition s' 3 =^c V3 f° r same probability 
space 7-3'. Moreover, there is a probability space V 3 s such that 

V'l= E H',s[P]V and Vi^V'is- (8-69) 

Let W2 be the weight function for V 2 ^n 2 ^3 s- For each probability space V of S7 2 5, let 
w-p : states(M 2 ) X Probs(states(M 3 )) —^ [0, 1] be a function that is non-zero only in the set 
X O3 5 and such that for each pair (s,V) of S7 X Sl"^, 

., P\s]w 2 (s,T") 

Wt(s > v)= i%\- (8J0) 

Also, for each probability space V of Q,' 2 s , let 

^5 = E E M^OW), (8-71) 

and let 

Vl = E ^st^'- (8-72) 

Let Pg 5 be the discrete probability space where S7 3 s = {V 3 \ V G ^2,s} 5 an d for each element 
7- > 3 > of S7 3 5 , P3 5[7 7 3 P ] = J2-pi G n' \-pv--pV P 2 st^ 7 ']- Then, the following properties are true. 

1. For each probability space V of fl' 2 s , w-p is a weight function from V to V 3S . 

We verify separately each one of the conditions that a weight function must satisfy. 
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(a) For each s G states(M 2 ), P[s] = J2v'eProbs(states(M 3 )) w-p{s,V). 

From the definition of w-p, the right expression above can be rewritten into 

E ^p- <«•"> 

■p'ePTOis(s<a<es(M 3 )) 2 L J 

Since w 2 is a weight function, J2v'eProbs(states(M 3 )) w 2(s,T") = P^[s], and thus Ex- 
pression 8.73 becomes P[s]. 

(b) For each V G Probs(.states(M 3 )), Zs€,tates{M 2 ) M^O = P£ S [V]. 

From Equation (8.71), P 3 s[P'] = J2 s e^ w T : '( s ^')- Since w-p is non-zero only when 
the first argument is in fi, P 3 ^ 5 [P'] = E ses < a <e S (M 2 ) w v (s,T"). 

(c) For each (s,P') G states(M 2 ) X Probs(states(M 3 )), if w v (s,V') > then s ^ 2 T 7 '. 
If w-p(s,V') > 0, then, from Equation (8.70), W2(s,V') > 0. Since wi is a weight 
function, then s 1Z 2 V . 

2- Y.ve^ s PU V ^ V = V "- 

From the definition of V 3 s , Equation (8.72), Equation (8.71), and Equation (8.70), 
J2veQ' -^3 s\P\P can be rewritten into 

E E E K S tPi m ^:; V ' ] V. (8.74) 

From (8.68), Expression (8.74) can be rewritten into 

E E «^V- (8.75. 

V'ESl" s sEstates(M 2 ) 2 L J 

After simplifying -Pjfs], s i nce w 2 is a weight function from V' 2 to P" s , Expression (8.75) 
can be rewritten into 

£ KsW, (8-76) 

which can be rewritten into V 3 using Equation (8.69). 

3. For each pair (s^, V) such that $[ 1Z\ V, s[ IZ3 V 3 . 
This follows directly from 1 and (8.72). 

Let V 3 be V3, and define a new weight function w : states(M\) X Probs(states(M 3 )) —^ [0,1] 
such that, for each probability space V of fl' 2 s , w(si,V 3 ) = w\( < s\,V). Then, it is easy to check 
that V' x Ck V 3 s via w. This fact, together with 2, is sufficient to complete the proof. ■ 
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8.7 Probabilistic Forward Simulations and Trace Distributions 

In this section we show that probabilistic forward simulations are sound for the trace distribution 
precongruence. Specifically, we show that Mi C_fs M 2 implies M\ C.£> M 2 . Thus, since C_fs is 
a precongruence that is contained in C£>, from the definition of C_dc we obtain that M\ C_fs M 2 
implies Mi C_dc M2. 

Proposition 8.7.1 Let M x Q FS M 2 . Then M x Q D M 2 . 

Proof. Let 1Z be a probabilistic forward simulation from Mi to M 2 , and let Hi be a proba- 
bilistic execution of Mi that leads to a trace distribution T>\. From Lemma 8.6.1, there exists 
a probabilistic execution H 2 of M 2 and two mappings m, S such that (Hi, H 2 , m, S) is an exe- 
cution correspondence structure for 1Z. We show that H 2 leads to a trace distribution V 2 that 
is equivalent to T>\. 

Consider a cone Cp of T>\ . The measure of Cp is given by 

E PhACJ- (8.77) 

qi £states(Hi)\trace(qi)=/3,lact(qi) = lact(/3) 

The same value can be expressed as 

lim J2 PhACiA- (8-78) 

% — >oo 

<3l (zfringe(Hi ,i)\f3< trace (qi) 

Consider a cone Cp of X>2- The measure of Cp is given by 

E PnAC q A- (8-79) 

q2£states(H2)\trace(q2)=i3,Iact(q2) = lact(j3) 

The same value can be expressed as 

lim E Pm{i)[C q A- (8-80) 

1 — >oo — 

g2em(8)|/3<<race(g 2 ) 

The reason for the alternative expression is that at the limit each cone of Expression (8.79) is 
captured completely. Thus, it is sufficient to show that for each finite (3 and each i, 

E PhAC*A= E Pm(i)[Q2]. (8-81) 

<3l (^fringe (Hi ,i)\/3<.trace(qi) q2€.m(i)\/3<trace(q2) 

This is shown as follows. Let W{ be the weight function for m(i) \Z-jz S(i). Then, 

E PhACA= E E MQiM- (8.82) 

q(zfringe(Hi ,i)\f3<.trace(q) q\ ^jringe{H\ ,i)\f3<. trace (qi) V2^zS(i) 

Observe that each probability space of S(i) has objects with the same trace, that each state q 
of fringe(Hi,i) is related to some space of S(i), and that each space of S(i) is related to some 
state q of fringe(Hi,i). Thus, from (8.82), 

E PH 1 [C q ]= E E <?i^2). (8.83) 

qEfringe(Hi,i)\f3<trace(q) T ? 2&S(i)\3 q2en2 fj<trace(q2) qi£fringe(Hi,i) 
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Since W{ is a weight function, we obtain 

£ PnAC q ]= £ Ps(i)[P2]. (8.84) 

qEfringe(Hi,i)\f3<trace(q) T ? 2&S(i)\3 q2en2 fj<trace(q2) 

Since in a probability space the probability of the whole sample space is 1, we obtain 

£ PnAC q ]= £ £ PswP^fe]- (8.85) 

qEfringe(H 1 ,i)\f3<trace(q) T , 2ES(i)\3 q2e Q 2 P<trace(q2) 926^2 

From an algebraic manipulation based on Condition 3 of an Execution Correspondence Struc- 
ture, we obtain 

£ P Hl [C q ]= £ £ PswP^fe]- (8.86) 

qEfringe(H 1 ,i)\f3<trace(q) q2Em(i)\f3<trace(q2) ^GS^') 1 92 6^2 

Finally, from Condition 3 of an Execution Correspondence Structure again, we obtain Equa- 
tion (8.81). ■ 

8.8 Discussion 

Strong bisimulation was first defined by Larsen and Skou [LS89, LS91] for reactive processes. 
Successively it was adapted to the alternating model by Hansson [Han94]. In this thesis we 
have defined the same strong bisimulation as in [Han94]. The formal definition differs from the 
definition given by Hansson in that we have used the lifting of a relation to probability spaces 
as defined by Jonsson and Larsen [JL91]. 

Strong simulation is similar in style to the satisfaction relation for the probabilistic specifi- 
cation systems of Jonsson and Larsen [JL91]. It is from [JL91] that we have borrowed the idea 
of the lifting of a relation to a probability space. 

The probabilistic versions of our simulation relations are justified both by the fact that a 
scheduler can combine transitions probabilistically, as we have said in this thesis, and by the fact 
that several properties, namely the ones specified by the logic PCTL of Hansson and Jonsson 
[Han94], are valid relative to randomized schedulers iff they are valid relative to deterministic 
schedulers. This fact was first observed by Segala and Lynch [SL94] and can be proved easily 
using the results about deterministic and randomized schedulers that we proved in Chapter 5. 

The weak probabilistic relations were introduced first by Segala and Lynch [SL94]. No 
simulation relations abstracting from internal computation were defined before. Probabilistic 
forward simulations are novel in their definition since it is the first time that a state is related 
to a probability distribution over states. 
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Chapter 9 

Probabilistic Timed Automata 

9.1 Adding Time 

So far we have extended labeled transition systems to handle probabilistic behavior; however, 
we have not addressed any real-time issue yet. The main objective of this chapter is to add 
time to probabilistic automata. 

Following an approach that Abadi and Lamport [AL91] call the "old-fashioned recipe", we 
address real-time issues by augmenting probabilistic automata with some structure that models 
passage of time. In particular, we adopt the solution of Lynch and Vaandrager [LV95], where 
a timed automaton is an ordinary automaton whose actions include the positive real numbers. 
The occurrence of a real number d means that time d elapses. In addition, a timed automaton 
of [LV95] is required to satisfy two trajectory axioms: the first axiom says that if time d can 
elapse and immediately afterwards time d! can elapse, then time d + d! can elapse; the second 
axiom says that if time d can elapse, then there is a trajectory that allows us to associate every 
real time in the interval [0, d] with a state. 

The introduction of real-time in probabilistic automata presents two main problems. 

1. Time is a continuous entity, and the time that elapses between the occurrence of two sep- 
arate actions may depend on a probability distribution that is not discrete. For example, 
the response time of a system may be distributed exponentially. On the other hand, the 
probability distributions that we allow in the untimed model are only discrete. 

2. In the untimed model the parallel composition operator is defined only for simple prob- 
abilistic automata. Since time-passage is modeled by actions of !R + , in a simple proba- 
bilistic timed automaton it is not possible to let time pass according to some probability 
distribution. 

The first problem could be solved by removing the requirement that the probability distribution 
associated with a transition is discrete. However, in such case we would need to redevelop the 
whole theory, while if we force each probability distribution to be discrete we can reuse most 
of the results of the untimed model. For this reason, we choose to work only with discrete 
probability distributions and we defer to further work the extension of the model to non-discrete 
probability distributions (cf. Section 13.2.1). 
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For the second problem the reader may object that it originates from the choice of using 
a distinct time-passage action for each amount of time that elapses in a transition, and thus 
we may conclude that the problem would be solved by using a unique action that expresses 
passage of time [LV93b] rather than a different action for every time; however, the problem has 
deeper roots. 

Example 9.1.1 (Problems with probabilistic passage of time) Suppose that from state 
si a probabilistic timed automaton Mi lets time pass for 1 second with probability 1/2 and for 
2 seconds with probability 1/2 before performing an action a, and suppose that from state «2 a 
probabilistic timed automaton Mi lets time pass for 0.5 seconds with probability 1/2 and for 1.5 
seconds with probability 1/2 before performing action a. What is the probability distribution 
on the time that elapses from state (si,S2) of Mi||M2 before performing a? What can we 
say about the projections of a probabilistic execution of Mi||M2? The reader may note the 
similarity with the problems encountered in the definition of parallel composition for general 
probabilistic automata (cf. Section 4.3.3). ■ 

In order to simplify the handling of trajectories, in this thesis we impose an additional restric- 
tion on the time-passage transitions of a probabilistic timed automaton; namely, each transition 
involving time-passage is required to lead to a Dirac distribution. Probabilistic behavior as- 
sociated with passage of time is allowed only within a probabilistic execution. Even though 
this timed model may appear to be restrictive, it is sufficiently powerful to analyze non-trivial 
timed properties of randomized algorithms (cf. Chapter 10). 

In the rest of this chapter we concentrate on the definition of the timed model as an extension 
of the probabilistic automata of Chapter 4. Most of the concepts are extensions of the definitions 
of [LV95] to the probabilistic framework; the non-trivial part of the chapter is the definition of 
a probabilistic timed execution, where some measure-theoretical complications arise. 

9.2 The Timed Model 

In this section we define probabilistic timed automata as an extension of the probabilistic 
automata of Chapter 4, and we extend the timed executions of [LV95] to our framework. Due 
to the complications that arise in the definition of a probabilistic timed execution, we define 
probabilistic timed executions in a separate section. 

9.2.1 Probabilistic Timed Automata 

A probabilistic semi-timed automaton M is a probabilistic automaton whose set of external 
actions includes !R + , the set of positive reals, and whose transitions with some action in !R + 
are non-probabilistic, i.e., they lead to a Dirac distribution. Actions from !R + are referred to as 
time-passage actions, while non-time-passage actions are referred to as discrete actions. We let 
d, d' , . . . range over !R + and more generally, t, t' , . . . range over the set ?R. U {oo} of real numbers 
plus infinity. The set of visible actions is defined by vis(M) = ext(M) \ ?R. + . 

A probabilistic timed automaton is a probabilistic semi-timed automaton M that satisfies 
the following two axioms. 

Al If s — ► s' and s' — ► s", then s — ► s". 
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For the second axiom, we need an auxiliary definition of a trajectory , which describes the 
state changes that can occur during time-passage. Namely, if / is any left-closed interval of ?R. 
beginning with 0, then an I-trajectory is a function uj : / — ► states(M), such that 

uj(t) -4 uj(t') for all t,t' e I with t < t' . 

Thus, a trajectory assigns a state to each time t in the interval /in a "consistent" manner. We 
define Itime(uj), the "last time" of a;, to be the supremum of I. We define fstate(uj) to be w(0), 
and if / is right-closed, we also define Istate(uj) to be uj(ltime(uj)). A trajectory for a transition 

s — ► s' is a [0, <i]-trajectory such that fstate(uj) = s and Istate(uj) = s' . Now we can state the 
second axiom. 

A2 Each time-passage transition s — ► s' has a trajectory. 

A probabilistic timed automaton M is simple if M is a simple probabilistic automaton. 

Axioms Al and A2 express natural properties of time: Axiom Al says that if time can 
elapse in two transitions, then it can also elapse in a single transition; Axiom A2 says that if 
time d can elapse, then it is possible to associate states with all times in the interval [0,d] in a 
consistent way. 

Example 9.2.1 (The patient construction) A simple way to add time to a probabilistic 
automaton is to add arbitrary self-loop timed transitions to each state of a probabilistic au- 
tomaton. Specifically, given a probabilistic automaton M, we define patient(M) to be the 
probabilistic timed automaton M' such that 

1. states(M') = states(M), 

2. start(M') = start(M), 

3. acts(M') = acts(M) U K+, 

4. trans(M') = trans(M) U {(s, d,s)\s e states(M), d G K+}. 

Thus, patient(M) is like M except that an arbitrary amount of time can elapse between two 
discrete transitions. It is immediate to verify that patient(M) satisfies axioms Al and A2. 
The patient construction was first defined for ordinary automata in [VL92]. ■ 

Example 9.2.2 (Simple restrictions on time passage) The patient construction does not 
specify any limitations to the way time can elapse. Sometimes we may want to specify upper 
and lower bounds to the time it takes for some transition to take place. Such a limitation can 
be imposed easily by augmenting the states of a probabilistic automaton with variables that 
express the time limitations that are imposed. As an easy example consider a probabilistic 
automaton M with a unique state s and a unique discrete transition (s,a,s). Suppose that we 
want to add time to M and impose that action a occurs once every at least 1 time unit and at 
most 2 time units. Then the corresponding probabilistic timed automaton M' can be specified 
as follows. 

1. states(M') = {(s, /, h) \ < / < 1, < / < h < 2}, 
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2. start(M') = {(s,0,2)}, 

3. acts(M') = {a}U»+, 

4. trans(M') = {((s, 0, h), a, (s, 1, 2)) | < /i < 2} U {((s, /, /i), d, (s, / - rf, /i - dj) \ d < I < 
h} U {((s, 0, /i), rf, (s, 0, h - d))d < h}. 

The variables / and h keep track of the time that must or can elapse before performing a. Time 
passage decreases both the variables unless they are 0. Action a can occur only when / = 
and leads to a state where 1 = 1. This means that at least 1 time unit must elapse before a 
can be performed again. No time can elapse if h = 0. At thet point the only transition that 
can be performed is the transition labeled with a. Thus, no more than 2 time units can elapse 
between the occurrence of two actions a. It is immediate to verify that M' satisfies axioms Al 
and A2. ■ 

9.2.2 Timed Executions 

Since a probabilistic timed automaton is also a probabilistic automaton, the executions of the 
untimed model carry over to the timed case. However, an execution associates states with just 
a countable number of points in time, whereas the trajectory axiom A2 allows us to associate 
states with all real times. Also, our intuition about the executions of a timed system is that 
visible actions occur at points in time, and that time passes "continuously" between these 
points. In other words, at each point in time a system is in some state. This leads to the 
definition of a timed execution. 

Timed Executions 

A timed execution fragment a of a probabilistic timed automaton M is a finite or infinite 
alternating sequence, a = cJoai^ia2^2 • • •, where 

1. Each oji is a trajectory and each a 8 - is a discrete action. 

2. If a is a finite sequence then it ends with a trajectory. 

3. If oji is not the last trajectory in a then its domain is a right-closed interval, and there 
exists a transition (lstate(ui),V) of M such that (a,fstate(uji + i)) £ S7. 

A timed execution fragment describes all the discrete changes that occur, plus the evolution 
of the state during time-passage transitions. If a is a timed execution fragment, then we 
let Itime(a) denote J2i Itime(uji). Note that we allow the case where the domain of the final 
trajectory is of the form [0,oo); in this case Itime(a) = oo. We define the initial state of a, 
fstate(a), to be fstate(ujo) 

A timed execution is a timed execution fragment whose first state is a start state. 

The timed executions and timed execution fragments of a probabilistic timed automaton 
can be partitioned into finite, admissible, and Zeno timed executions and timed execution 
fragments. A timed execution (fragment) a is finite, if it is a finite sequence and the domain of 
its final trajectory is right-closed; a timed execution (fragment) a is admissible if Itime(a) = oo; 
a timed execution (fragment) a is Zeno if it is neither finite nor admissible. 
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There are basically two types of Zeno timed executions: those containing infinitely many 
discrete actions in finite time, and those containing finitely many discrete actions and for which 
the time interval associated with the last trajectory is right-open. Thus, Zeno timed executions 
represent executions of a probabilistic timed automaton where an infinite amount of activity 
occurs in a bounded period of time. (For the second type of Zeno timed executions, the infinitely 
many time-passage transitions needed to span the right-open interval should be thought of the 
"infinite amount of activity".) 

We will be interested mostly in the admissible timed executions of a probabilistic timed 
automaton since they correspond to our intuition that time is a force beyond our control that 
happens to approach infinity. However, according to our definition of a probabilistic timed 
automaton, it is possible to specify probabilistic timed automata in which from some states 
no admissible timed execution fragments are possible. This can be because only Zeno timed 
execution fragments are possible from that state, or because time cannot advance at all (in which 
case a time deadlock has occurred). Although Zeno timed executions are usually non- desirable, 
research experience has shown that the analysis of a model would be more complicated if Zeno 
timed executions are ruled out. 

Denote by t-frag*(M), t-frag°°(M), and t-frag(M) the sets of finite, admissible, and all 
timed execution fragments of M. Similarly, denote by t-exec*(M), t-exec°°(M), and t-exec(M) 
the sets of finite, admissible, and all timed executions of M. 

A timed extended execution fragment of M, denoted by a, is either a timed execution 
fragment of M or a sequence a' 8 where a' is a timed execution fragment of M. Denote by 
t-exec* s (M) and t-exec$(M) the sets of finite and all timed extended executions of M. 

Concatenations, Prefixes and Suffixes 

If uj is an /-trajectory where / is right-closed, and uj' is an /'-trajectory such that Istate(uj) = 
fstate(uj'), then uj and uj' can be concatenated. The concatenation, denoted by ujuj' is the least 
trajectory (the trajectory with the smallest domain) uj" such that uj"(t) = uj(t) for t G /, and 
uj"(t + Itime(uj)) = tjj(t) for t G /'. It is easy to show that uj" is a trajectory. 

Likewise, we may combine a countable sequence of "compatible" trajectories into one: if uj{ 
is an /-trajectory, < i < oo, where all /■ are right-closed, and if Istate(uji) = fstate(uji + i) for 
all i, then the infinite concatenation uj\UJi • • • is the least function uj such that for all i and all 
t G /', oj(t + J2j<i Itime(ujj)) = uJi(t). It is easy to show that uj is a trajectory. 

A finite timed execution fragment a = uJoaiUJi ■ ■ ■ a n uj n of M and a timed (extended) execu- 
tion fragment a' = uj' n a n+ iuj n+ i ■ ■ ■ of M can be concatenated if Istate(a) = fstate(a'). In this 
case the concatenation, written a~ a' , is defined to be a" = UJoaiUJi ■ ■ •a n (cj n cj^)a n _|_icj n _|_i • • •. 
It is easy to see that a is a timed (extended) execution fragment of M. 

The notion of prefix for timed execution fragments and timed extended execution fragments 
is defined as follows. A timed (extended) execution fragment a of M is a prefix of a timed 
(extended) execution fragment a' of M, written a < a', if either a = a' or a is finite and there 
exists a timed (extended) execution fragment a" of M such that a' = a ~ a". Likewise, a is a 
suffix of a' if there exists a finite timed execution fragment a" such that a' = a" ~ a. Denote 
a by a'>a". 

The length of a timed execution fragment a expresses the number of discrete actions in 
a. Thus, even though a is admissible or Zeno (and thus not finite), its length may be finite. 
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Formally, define the length of a = cJoai^ia2^2 • • • as 

, , a \ n if a is a finite sequence and ends in u> n 
\ol\ = \ 

1 oo if a is an infinite sequence. 

9.3 Probabilistic Timed Executions 

Since a probabilistic timed automaton is also a probabilistic automaton, it is possible to talk 
about the probabilistic executions of a probabilistic timed automaton. However, as we have 
pointed out already for ordinary executions, a probabilistic execution does not describe com- 
pletely the evolution of a probabilistic timed automaton since it does not allow us to associate 
every real time with the states that are reached at that time. We need a structure that extends 
probabilistic executions in the same way as a timed execution extends an execution. A timed 
execution differs from an execution in two aspects: 

1. a timed execution has trajectories to express passage of time; 

2. a timed execution does not contain any time-passage actions. 

In particular, a timed execution hides the time-passage transitions that are scheduled in an 
execution to let time pass. Given a trajectory u, there are infinitely many ways to schedule time- 
passage transitions to move in time Itime(uj) from fstate(uj) to Istate(uj) (Istate(uj) is meaningful 
only if the domain of uj is right-closed); the trajectory uj represents all those possible ways. In a 
similar way, a probabilistic timed execution should not contain any information on the specific 
time-passage transitions that are scheduled. Thus, a probabilistic timed execution should be 
a structure where each state records the past history and each transition contains information 
on the trajectories that are spanned till the occurrence of the next action. However, it may be 
the case that there is no next action since the next trajectory is right-open. This would not 
be a problem except for the fact that from a state there can be uncountably many right-open 
trajectories that leave even though they are generated by scheduling time-passage transitions 
according to a discrete probability distribution. 

Example 9.3.1 (Uncountable branching from countable branching) Consider a prob- 
abilistic automaton M that can increase or decrease a variable x of its state at a constant speed, 
and suppose that every one time unit the speed of x can be complemented nondeterministi- 
cally. A valid scheduler A for M is a scheduler that every one time unit chooses the sign of the 
speed of x according to a uniform binary distribution. As a result, there are uncountably many 
trajectories leaving from the start state of M if we use A to resolve the nondeterminism. Thus, 
if in a probabilistic timed execution we do not allow for a trajectory to be split into pieces, 
the probabilistic timed execution of M generated by A would have a non-discrete probability 
distribution in its transition relation. ■ 

To express the fact that we allow only discrete probability distributions on a scheduler, we define 
probabilistic timed executions in two steps. First we define probabilistic time-enriched execu- 
tions, which contain closed trajectories and time-passage actions (the time-passage transitions 
that are scheduled are visible); then, we remove the time-passage actions from probabilistic 
time-enriched executions to yield probabilistic timed executions. 
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At the end of this section we show that probabilistic executions, probabilistic time-enriched 
executions, and probabilistic timed executions are strongly related. Specifically, we show that 
each probabilistic execution is a sampling of a probabilistic time-enriched execution where 
the information contained in the trajectories is lost, and that each probabilistic time-enriched 
execution is sampled by some probabilistic execution. Furthermore, we show that it is possible to 
define an equivalence relation directly on probabilistic time-enriched executions that expresses 
the fact that two probabilistic time-enriched executions denote the same probabilistic timed 
execution (they just schedule time-passage transitions in a different way). 

All the equivalence results that we prove in this section allow us to use the kind of proba- 
bilistic execution that is best suited for each problem. In particular, we use probabilistic timed 
executions for the theorems of Chapter 10, and we use probabilistic time-enriched executions 
and probabilistic executions for the results of Chapters 11 and 12. Due to the purely technical 
content of the comparison section (Section 9.3.3), the reader may focus just on the definitions 
and on the informal explanations (Sections 9.3.1 and 9.3.2) at a first reading. Most of the 
concepts are simple modifications of concepts defined for probabilistic executions. 

9.3.1 Probabilistic Time-Enriched Executions 

Time-Enriched Executions 

Let M be a probabilistic timed automaton. A time- enriched execution fragment of M is a finite 
or infinite alternating sequence a = cJoai^ia2^2 • • • where 

1. The domain of ujo is [0,0]. 

2. Each oji is a trajectory with a closed domain and each a 8 - is an action. 

3. If ai is a visible action, then the domain of LO{ is [0,0], and there exists a transition 
(lstate(ui-i),V) of M such that (a,-, fstate (a;,-)) G 0. 

4. If ai is a time-passage action, then the domain of LO{ is [0, a 8 ] and lstate(oJi_i) = fstate(uji). 

Denote by te-frag*(M) and te-frag(M) the set of finite and all time-enriched execution fragments 
of M, respectively. The notation for fstate(a), Istate(a) and Itime(a) extends trivially. 

A time-enriched execution fragment a contains more information than a timed execution 
fragment since it is possible to observe what time-passage transitions are used to generate a. 

A time-enriched extended execution fragment of M is either a time-enriched execution frag- 
ment of M or a sequence a 8 where a is a finite time-enriched execution fragment of M. The 
notation for Istate(a) extends trivially. 

A finite time-enriched execution fragment a = o^ai^i • • • a n uj n of M and a time-enriched 
extended execution fragment a' = u' n a n +iu n +i ■ ■ ■ of M can be concatenated if Istate(a) = 
fstate(a'). In this case the concatenation is defined to be a" = o^ai^i • • ■ a n uj n a n+ iuj n+ i ■ ■ •, 
and is denoted by a" a'. It is easy to see that a" is a time-enriched extended execution 
fragment of M. A time-enriched extended execution fragment a of M is a prefix of a time- 
enriched extended execution fragment a' of M, written a < a', if either a = a' or a is finite 
and there exists a time-enriched extended execution fragment a" of M such that a' = a ~ a". 
Likewise, a is a suffix of a' if there exists a finite time-enriched execution fragment a" such 
that a' = a" ~ a. Denote a by a'>a". 
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Time-Enriched Transitions 

Let (s,V) be a combined transition of M. For each pair (a,s') of 0, if a is a discrete action, 
then let Vt a ^ s i\ be V((a,s')); if a is a time-passage action, then let Vt a ^ s i\ be a discrete proba- 
bility distribution of Probs(trajectories(M,s, a, s')), where trajectories(M,s,a,s r ) denotes the 
set of trajectories for s — ► s' . The pair J2( a s ')efi -f[( a ? s ')]( s '^ :> (a,s')) * s called a time- enriched 
transition of M. 

Thus, a time-enriched transition adds information to a combined transition by specifying 
what state is reached at each intermediate time. A combined transition gives just the extremes 
of a trajectory, dropping all the information about what happens in the middle. 

Probabilistic Time-Enriched Executions 

A probabilistic time- enriched execution fragment H of a timed probabilistic automaton M is a 
fully probabilistic automaton such that 

1. states(H) C te-frag*(M) 

2. for each transition tr = (q,V) of H there is a time-enriched transition tr' = (lstate(q),V r ) 
of M, called the corresponding time-enriched transition, such that V = q ~ V . 

3. each state of H is reachable and enables one transition. 

A probabilistic time-enriched execution is a probabilistic time-enriched execution fragment 
whose start state is a start state of M. Denote by te-prfrag(M) the set of probabilistic time- 
enriched execution fragments of M, and by te-prexec(M) the set of probabilistic time-enriched 
executions of M. Also, denote by q$ the start state of a generic probabilistic time-enriched 
execution fragment H . 

As for the untimed case, there is a strong relationship between the time-enriched extended 
execution fragments of a probabilistic timed automaton and the extended executions of one of 
its probabilistic time-enriched execution fragments. Specifically, let M be a probabilistic timed 
automaton and let if be a probabilistic time-enriched execution fragment of M. Let go be the 
start state of H . For each extended execution a = go a i?i ■ ■ ■ of H , let 

a J qo ~ lstate(qo)ailtraj (qi)a2 • • • if a does not end in 6, , . 

1 go ~ lstate(qo)ailtraj(qi)a2- ■ •a n ltraj( < q n )8 if a = qoa\qi ■■■ a n q n 6 , 

where Itraj(qi) denotes the last trajectory of qj. It is immediate to observe that a{ is a time- 
enriched extended execution fragment of M. For each time-enriched extended execution frag- 
ment a of M such that go < «, i.e., a = go ~ ^o«i^i • • •, let 

a J goai(go«i^i)«2(?o a i w i a 2^2) • • • if a does not end in 6, , . 

\ qoai(qoaiUi) ■ ■ ■ (qoaiui ■ ■ ■ a n u n )6 if a = g ai^i • • -a n cj n S. 

It is immediate to observe that a|go is an extended execution of some probabilistic timed 
execution fragment of M. Moreover, the following proposition holds. 



202 



Proposition 9.3.1 Let H be a probabilistic time-enriched execution fragment of a probabilistic 
timed automaton M. Then, for each extended execution a of H , 

(aj)|go = «, (9.3) 

and for each time-enriched extended execution fragment a of M starting with go, 

(atgo)i = "• (9.4) 

Events 

The probability space Vh associated with a probabilistic time-enriched execution H is defined 
as for the untimed case. Thus, £l' H is the set of time-enriched extended execution fragments of 
M that correspond to complete extended executions of H , i.e., 

£l' H = {aj, | a is a complete extended execution of H}, (9-5) 

where an extended execution a of if is complete iff either a is infinite, or a = a'S, a' is a finite 
execution of H , and 6 G £l¥ 4 4 < \. For each finite time-enriched extended execution fragment 

' Istate(a) ° 

a of M, let C^ denote the cone 

C% = {a' e tt H I a < a'}. (9.6) 

Let Ch be the set of cones of H . Then define T' H to be the u-field generated by Ch, i.e., 

T'u = °(Ch). (9.7) 

Define a measure fi on Ch such that the measure iih(C^) of a cone C„ is the product of the 
probabilities associated with each edge that generates a in if. Formally, let go be the start 
state of if . If a < go, then 

fiH(C%) = 1; (9.8) 

if a = go ~ ^o«i^i • • -w n _ia n cj n , then 

/x ff (C^) = <[(ai,gi)]---P g UK,^)], (9.9) 

where for each i, 1 < i < n, qi = go ~ ^o«i^i • • -u^-iaj-u;,-; if a = go ~ ^o«i^i • • -u n -ia n u n 8, 
then 

/x ff (C^) = P q H [(a 1 ,q 1 )]---P q H n _ 1 [(a n ,q n )]P qn [S], (9-10) 

where for each i, 1 < i < n, qi = go ~ wo^i^i • • ■Ui-iaiUi. Then the probability measure P^ is 
the unique measure on .F# that extends //#, and Vh is the completion of Vh- 

Finite Probabilistic Time-Enriched Executions, Prefixes, Conditionals, and Suffixes 

Since a probabilistic time-enriched execution is a fully probabilistic automaton, the definitions 
of finiteness, prefix, conditional and suffix of Section 4.2.6 extend directly: we just need to 
define the length of a time-enriched execution fragment a as the number of actions that occur 
in a. 
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9.3.2 Probabilistic Timed Executions 

We now define the probabilistic timed executions of a probabilistic timed automaton. We 
use probabilistic time-enriched executions to characterize those transitions that originate from 
discrete schedulers. 

Timed Transitions 

A timed transition expresses the result of choosing either an infinite trajectory or a finite 
trajectory followed by some discrete action at random. However, a timed transition should 
be the result of scheduling a collection of time-enriched transitions, so that we are guaranteed 
that it is due to a discrete scheduler. For this reason, we derive a timed transition from the 
probability distribution associated with a time-enriched probabilistic execution. The derivation 
proceeds in two steps: first all the time-passage actions are removed and the corresponding 
trajectories are concatenated; then the resulting structure is truncated at the occurrence of the 
first action. 

Removing Time-Passage Actions. Let a = cJoai^ia2^2 • • • be a time-enriched execution 
fragment of a probabilistic timed automaton M. The timed execution represented by a, denoted 
by t-exec(a), is the sequence obtained from a by removing all the time-passage actions and by 
concatenating ah the trajectories whose intermediate action is removed. 

Let if be a probabilistic time-enriched execution fragment of a probabihstic timed automa- 
ton M. Let 

J7 = t-exec(Q,H) U Umits(t-exec(Q,H)), (9-H) 

where Umits(t-exec(Q,H)) is the set of timed executions a of M that end with an open trajectory 
and such that for each finite prefix a' of a there is an element a" of t-exec(Q,H) such that a' < a". 
Then, t-exec{Vjj) denotes the probability space completion((Q,,J-,P)) where T is the u-field 
generated by the cones on 0, and P is t-exec^Pjj)- 

The reason for the definition of the sample space of t-exec^Pfj) is mainly technical: we 
want to establish a relationship between probabilistic time-enriched executions and probabilis- 
tic timed executions, and we want the relationship to be preserved by projection of probabilistic 
timed executions in a parallel composition context. Informally, we are interested in a distribu- 
tion over trajectories, possibly followed by an action, without keeping any information on how 
such a distribution is obtained. The elements of the sample space that end with right open 
trajectories can be affected by the way the transitions are scheduled in a probabilistic time- 
enriched execution. Moreover, these elements of can create problems for parallel composition. 
Closing the sample space under limit makes such differences invisible. The reader interested in 
more details is referred to Sections 9.3.3 and 9.5, and specifically to Examples 9.3.3 and 9.5.1. 

Example 9.3.2 (What t-exec identifies) Figure 9-1 gives an example of two probabilistic 
time-enriched executions that are mapped to the same structure by t-exec(). We assume to 
have two functions uj and u/ defined on the real numbers, and we denote by u^d' the trajectory 
oj" with domain [0, d! — d] such that for each t < d! — d, w"(t) = u(t — d). A similar notation is 
used for uj' . ■ 
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Figure 9-1: Probabilistic time-enriched executions that are mapped to the same structure. 

Truncation at the First Action. Let M be a probabilistic timed automaton, and let q be 
a finite timed execution fragment of M. For each extended timed execution fragment a of M 
such that q < a, let 



a if no action occurs in a\>q 

q ~ tjjoaifstate(oji) if at>g = o^ai^i 



truncate q (a) = \ ^ n w ^ , ( (9.12) 



Let if be a probabilistic time-enriched execution fragment of M, and let g be a prefix of 
the start state of H . Then define truncate q {t-exec{Vu)) to be the probability space 7-* where 
J7 = truncate q (t-exec(£lH)), T is the u-field generated by the cones of 0, and P is the measure 
truncate q (t-exec(PH))- 

Timed Transitions. A timed transition of M leaving from a state s is a pair (s,V) such 
that there is a probabilistic time-enriched execution fragment H of M starting in s, and V = 
truncate s ( t-exec{Vjj) ) ■ 

Probabilistic Timed Executions 

A probabilistic timed execution fragment of a probabilistic timed automaton M, denoted by H, 
consists of four components. 

1. A set states(H) C t-frag s (M) of states. 

2. A unique start state q$ . 

3. An action signature sig(H) = sig(M). 

4. A transition relation trans(M) consisting of pairs (q,V) such that there exists a timed 
transition (lstate(q),V r ) of M satisfying V = q^V' . Observe that, from the discussion in 
Section 3.1.5, q ~ V is well defined. 

Moreover, each state of H is reachable, enables at most one transition, and enables one transition 
iff it is a finite timed execution fragment of M. A probabilistic timed execution of M is a 
probabilistic timed execution fragment of M whose start state is a start state of M . 

An execution of H is a sequence of states of 77, a = q^qi • • • , such that for each i, g 8 _|_i £ 0^. 
As for the untimed case, there is a strong correspondence between the timed extended execution 
fragments of a probabilistic timed execution H of M and the executions of H . Specifically, let 
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M be a probabilistic timed automaton and let if be a probabilistic timed execution fragment 
of M. Let go be the start state of H . For each execution a = gogi ■ ■ ■ of H , let 

a{ = limg 8 , (9.13) 

i 

where the limit is taken under prefix ordering. It is immediate to observe that a{ is a timed 
extended execution fragment of M. For each timed extended execution fragment a of M such 
that go < «, i.e., a = go ~ ^o«i^i • • •, let g 8 - be go ~ ^o«i^i • • • aifstate(uji), and if a>go is a finite 
sequence with n discrete actions, let g n +i be a. Then let 

a\qo = qoqiq2---- (9.14) 

It is immediate to observe that ajgo is an execution of some probabilistic timed execution 
fragment of M. Moreover, the following proposition holds. 

Proposition 9.3.2 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M. Then, for each execution a of H , 

(aj)|go = a, (9.15) 

and for each timed extended execution fragment a of M starting with go, 

(atgo)i = "• (9.16) 

Events 

The probability space Vh associated with a probabilistic timed execution fragment H is defined 
similarly to the untimed case. The set £l' H the set of extended timed execution fragments of 
M that correspond to complete executions of H , where an execution of H is complete iff it is 
either infinite or it leads to a state that does not enable any transition. The <7-field T' H is the 
minimum u-field that contains the class of cones of Cl' H . The measure P' H is the unique measure 
that extends the measure defined on cones as follows: if a = q$ ~ o^ai^i^ • • ~ a n^m then 

PnlCa] = <[?i] • ■■P? n _MP? n [C a ] (9.17) 

where for each i < n, qi = q$ ~ o^ai^i • • • a n fstate(ui); if a = q$ ~ cJoai^ia2 • • • (i n io n 6^ then 

P' H [C a ] = P?M ■ ■■ p Lii ( ln}P q H n [a} (9.18) 

where for each i < n, qi = q$ ^ cooaiiOi • • -a n fstate{ui). Observe that although there are 
uncountably many cones in T' H , every union of cones is expressible as a countable union of 
disjoint cones. Then, Vh is the completion of V H . 

Finite Probabilistic Timed Executions, Prefixes, Conditionals, and Suffixes 

Finiteness and prefix are defined similarly to the untimed case, and thus we do not repeat the 
definitions here. 

Conditionals and suffixes differ in a small detail concerning the start state. The reader 
should observe the similarity of these definitions to those for the untimed case. Also, observe 
that the properties of conditionals and suffixes (Propositions 9.3.3 and 9.3.4) are the same as 
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for the untimed case. This is what allows us to extend the results for the untimed case directly 
to the timed case. 

Let if be a probabilistic timed execution fragment of a probabilistic timed automaton M, 
and let q be a prefix of some state of H such that q$ is a prefix of q. Then H \q is a new 
probabilistic execution fragment defined as follows: 

1. states(H\q) = {q} U {q 1 G states(H) \ q < q'}; 

2. start(H\q) = {q}. 

3. for each state q' of H\q different from q, tr , = tr^f, . 

4. let q be the maximum state of H that is a prefix of q. Then, tr q = (q,V?\C q ). 

H | q is called a conditional probabilistic timed execution fragment. We show later that H \q is a 
probabilistic timed execution. Observe that ($lH\qi3~H\ q ,Pjj\ q ) and (0#|C g , Tu\C q , Pn\Cq) are 
the same probability space (cf. Section 3.1.8): the sample spaces are the same, the generators 
are the same, and the probability measures coincide on the generators. Thus, the following 
proposition is true. 

Proposition 9.3.3 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M , and let q be a prefix of a state of H such that q$ < q. Then, for each subset E 

°f^H\q, 

1- EeT H{q iffEeTn. 

2. If E is an event, then Pr[E] = P H [C q ]P H \ q [E]. ■ 

Let if be a probabilistic timed execution fragment of a probabilistic timed automaton M, and 
let q be a prefix of some state of H such that q$ is a prefix of q. Then H >q is a new probabilistic 
execution fragment defined as follows: 

1. states(H>q) = {q'oq \ q' G states(H\q)}; 

2. start(H\q) = {Istate(q)} . 

3. for each state q' of H>q, tr , >q = tr J 9 ,>o. 

i m q q q 

H\>q is called a suffix of if . It is easy to check that the probability spaces Vu>q an( i T^Hlq are 
in a one-to-one correspondence through the measurable function / : &H>q - ► ^H\q such that 
for each a G &H>qi f( a ) = q^ ot. The inverse of / is also measurable and associates a\>q with 
each timed execution a of £ljj\q- Thus, directly from Proposition 9.3.3, we get the following 
proposition. 

Proposition 9.3.4 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M , and let q be a prefix of a state of H such that q$ < q. Then, for each subset E 

Of &H>q, 

1. EeT H>q iff(q~E)ef H . 
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2. If E is an event, then Ph[q~ E] = PH[C q ]PH> g [E]. ■ 

We are left with showing that H \q is well defined. The proof of this apparently obvious fact is 
not simple and contains several technical details. 

Proposition 9.3.5 Let H be a probabilistic timed execution fragment of a probabilistic timed 
automaton M , and let q be a prefix of a state of H such that q$ < q. Then, H\q is a probabilistic 
timed execution fragment of M . 

Proof. We just need to verify that the transition leaving from state q in H\q is a timed 
transition. Let q be the maximum state of H that is a prefix of q. Then, from the definition 
of a timed transition, there is a probabilistic time-enriched execution fragment H q of M such 

that V^ = q ~ truncate i state iq\(t-exec(VHq))- From the definition of tr q , we need to find a 
probabilistic time-enriched execution fragment H q of M such that 

(<p truncate lsUte (q)(t-exec(V Hq )))\C q = q~ truncate lsUte{q) (t-exec(VH q ))- (9.19) 

Let q' be q\>q. From the definition of q, q' is just one closed trajectory. Thus, if we build H q 
such that 

(t-exec(P Hg ))\C q > = q'~ t-exec(P Hq ), (9.20) 

then Equation 9.19 follows easily using simple properties of truncate. Thus, the rest of this 
proof is dedicated to the construction of an H q that satisfies (9.20). 

Let qi,q2, ... be an enumeration of the minimal states q" of H such that q' < t-exec(q"). 
We distinguish two cases. 

1. For each i, t-exec(qi) = q'. 

The construction for H q in this case is carried out in the proof of Proposition 9.3.8 (cf. 
Equation 9.29). We give a forward pointer to avoid too many technical details at this 
point. 

2. There is an i such that q' < t-exec(qi). 

We prove this case by reducing the problem to the previous case. That is, we build a new 
probabilistic time-enriched execution fragment Hq such that t-exec{Vjjq) = t-exec(V}ji) 
and such that the minimal states q" of H' r such that q' < t-exec(q") satisfy q' = t-exec(q'). 

Recall first that q' is a trajectory whose domain is [0,d] for some d > 0. Define a 
collection of finite time-enriched execution fragments q[,q'2, • • • as follows: for each i, if 
t-exec(qi) = q' then q[ = qf, otherwise, represent qi as qi ~ lstate(qi)diUJi, where qi is 
a state of H q , and let q[ be qi ~ lstate{^qi)di^Ui^di^0Ji^di^0Ji^ where LO{ = 0^10^2^,3, 
t-exec(qi ~ Istateiq^di^uji^di^ifi) = <?', an d the actions di t \ and d{^ are chosen in such a 
way that for each i qi " lstate(qi)di t iUi t i is not a prefix of any of the </-'s, j 7^ i. In other 
words, we split all the g 8 's in such a way that a state that corresponds to q' is reached 
always and such that none of the states of H q are identified. Then, 

states(H'g) = {q" \ 3 t q" < q[} (9.21) 

U \\J{qi~ (q">Qi) I q" e states(H q ), qi < q"}\ . 
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The transition relation of H'g is obtained from the transition relation of Hq by scheduling 
the same time-enriched transitions of M as before except for the states qi where the 
intermediate transitions leading to the g''s are scheduled. It is simple to check that Hq 
satisfies the desired properties. ■ 

9.3.3 Probabilistic Executions versus Probabilistic Timed Executions 

In this section we show the relationship between probabilistic executions, probabilistic time- 
enriched executions, and probabilistic timed executions. The main idea is that they all repre- 
sent the same structures with different levels of detail. We show that a probabilistic execution 
is a sampling of a probabilistic time-enriched execution, where the information given by the 
trajectories is lost. Conversely, we show that each probabilistic time-enriched execution is 
sampled by some probabilistic execution. We show that each probabilistic time-enriched exe- 
cution represents a probabilistic timed execution and that each probabilistic timed execution 
is represented by some probabilistic time-enriched execution. Essentially, a probabilistic time- 
enriched execution is a probabilistic timed execution with the additional information of what 
time-passage transitions are scheduled. Finally, we define an equivalence relation on probabilis- 
tic time-enriched executions that captures the idea of representing the same probabilistic timed 
execution. This equivalence relation will be useful for parallel composition. 

Probabilistic Executions versus Probabilistic Time-Enriched Executions 

There is a close relationship between the probabilistic executions of a probabilistic timed au- 
tomaton and its probabilistic time-enriched executions. Informally, a probabilistic time-enriched 
execution contains more information than a probabilistic execution because it associates a state 
with every real time rather than with a countable set of times. In other words, a probabilistic 
execution can be seen as a sampling of a probabilistic time-enriched execution at countably 
many points. In later chapters we will see that probabilistic executions are sufficient for the 
study of the properties of a system whenever such properties do not depend on the actual states 
that are reached at each time. For the moment we just define what it means for a probabilistic 
execution to sample a probabilistic time-enriched execution, and we show that each probabilistic 
time-enriched execution is sampled by some probabilistic execution and that each probabilistic 
execution samples some probabilistic time-enriched execution. We start by defining a func- 
tion sample that applied to a probabilistic time-enriched execution if of a probabilistic timed 
automaton M gives a probabilistic execution H' of M, which by definition samples H . 

Let a = oJoaiOJia2tjJ2 • • • be a time-enriched execution of a probabilistic timed automaton 
M, and let sample(a) be the sequence a' = lstate(oJo)ailstate(oJi)a2lstate(oj2) • • •■ Then, it is 
easy to check that a' is an execution of M. We say that a' samples a. Define 

states(H') = sample(states(H)). (9.22) 

Let (q,V) be a transition of H . Define sample on as follows: sample((a,q')) = (a,sample(q')), 
and sample(S) = S. Then, define the transition sample((q,V)) to be 

sample((q,V)) = {sample(q),sample(V)). (9.23) 
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For each state q of if', let sample - (q) be the set of states q' of H such that sample(q') = 
q. Observe that all the states of sample -1 (q) are incomparable under prefix. For each q' £ 
sample~ (g), let 

sample- 1 (q) A Ph[C q '} ( . 

l J q' — y^ p ir< i' \y-^) 

2^ q"e sample- 1 (q) r H[^q"l 

Then, the transition enabled from q in H' is defined to be 

trf = J2 P q r' ple ~ 1(q) sample(trf). (9.25) 

q'tzsample - (g) 

Observe the similarity of Equations (9.24) and (9.25) with the equations that the fine the 
projection of a probabilistic execution (cf. Equations (4.21) and (4.22)). 

Proposition 9.3.6 below shows that H' is a probabilistic execution of M. We say that H' 
samples H. Then, Proposition 9.3.7 shows that each probabilistic execution samples some 
probabilistic time-enriched execution. 

Proposition 9.3.6 For each probabilistic time-enriched execution H of a probabilistic timed 
automaton M, sample(H) is a probabilistic execution of M . 

Proof. Let H' denote sample(H). The fact that each state of H' is reachable can be shown 
by a simple inductive argument; the fact that each state of H' is a finite execution fragment of 
M follows from a simple analysis of the definition of sample and of a time-enriched execution. 

We need to check that for each state q of H' the transition enabled from q in H' is generated 
by a combined transition of M. From (9.25), it is enough to show that for each state q' of 
sample - (q) the transition sample(tr^,) is generated by a combined transition of M. 

Since if is a probabilistic time-enriched execution of M, then there is a time-enriched 
transition (Istate(q')^V) of M such that Vzf = q' ~ V. From the definition of sample and the 
definition of a time-enriched transition, (Istate(q), sample(V)) is a combined transition of M, 
and sample(VW) = sample(q') ~ sample(V), which means that sample(V^ ) = q ~ sample(V). 
This is enough to conclude. ■ 

Proposition 9.3.7 Let H be a probabilistic execution of a probabilistic timed automaton M . 
Then there is a probabilistic time-enriched execution H' of M such that H = sample(H'). 

Proof. We build H' inductively in such a way that for each state q of H there is exactly one 
state q' of H' in sample - (q). The start state of H' is the same as the start state of H . 

Suppose that the transition relation of H' is defined for each state of length at most i — 1 
and assume that for each state q of H of length at most i there is exactly one state q' of H' in 
sample -1 (q). Let q be a state of H of length i and let q' be the state of sample -1 (q). Observe 
from the definition of sample that the length of q' is i. Let (lstate(q),V) be the combined 
transition of M that corresponds to tr^ . For each pair (a, s) of 0, if a is a discrete action, 
then let Vu^') be V((a,s')); if a is a time-passage action, then let Vu^') be V(w atS i), where 
w a,s' £ trajectories(M , s, a, s'). Let V' = E((is)efi^[( a ) s )]^(a,s)' Then, (lstate(q),V') is a time- 
enriched transition of M. Let tr q , be (q',q' ~ V). Then, tr q , is a legal transition for H'. 
Moreover, from the definition of i 7 ', each state of V^ is the sampling of exactly one state of 
V^ , and, vice versa, the sample of each state of V^ is a state of V^ . ■ 
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Probabilistic Time-Enriched Executions versus Probabilistic Timed Executions 

We define a function t-sample that, given a probabiiistic time-enriched execution fragment H 
of M, builds a probabilistic timed execution H' as follows. 

states(H') = {t-exec(q^) U (9.26) 

{q £ ^t-exec(H) I Q contains finitely many actions} U 
{q £ t-frag*(M) \ Itraj(q) is a [O,0]-trajectory and 3 q > e n t _ exec(H) q < q'}- 

The start state of H' is t-exec{(^\ and for each state q of H' the transition enabled from q is 
(q, truncate q (t- exec(T 'h)\C ' q )) ■ 

Proposition 9.3.8 t-sample(H) is a probabilistic timed execution fragment of M . 

Proof. We need to show that for each state q of H' that enables some transition there is 
a probabilistic time-enriched execution fragment H q of M starting from Istate(q) such that 

V q = truncate hute{q) (t-exec(V Hq ))- 

Let qi, q%, . . . be an enumeration of the states q' of H such that t-exec(q') = q, and for each 
i let pi denote Pjj\C q ^\. Observe that, since q ends with the occurrence of a discrete action, 
for each state q" of H such that q' < t-exec(q") there is an i such that qi < q". Define H q as 
follows. 

states(H q ) = [jstates(H>qi). (9.27) 

i 

For each state q' of H q , let 

H q a Y, % \ q >estates(H> qi ) P H[C qrql ]{tr qrq ,>qi) 






Z^j|g'es<aies(_ffl>g 8 ) -* H L^g, 

Then, it is enough to prove that 

q ~ t-exec(V Hq ) = t-exec(V H )\C q . (9.29) 

Before proving (9.29), we show the following property: for each state q' of H q , 

P Hq [C ql ] = ^^-^ . (9.30) 

This follows easily by induction using Equation (9.28) for the inductive step. The denominator 
is necessary for the base case to work. 

We now turn to Equation (9.29). Consider an extended timed execution fragment a of M, 
and distinguish the following two cases. 

1. a does not end with an open trajectory. 

Suppose that a £ ^t-exec(V H )\c ■ Then, from the definition of t-exec() and of the con- 
ditional operation, q < a and there is a time-enriched execution a' of 0# such that 
t-exec(a') = a. This means that there is a time-enriched execution a' of 0# such that 
t-exec(a') = a and there is a state qi of H such that qi < a' . From the construction of 
H q , each prefix of a 1 is a state of H q , and thus a' £ &t-exec(H )• The argument can be 
reversed. 
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2. a ends with an open trajectory. 

Suppose that a £ ^t-exec(P H )\c ■ Then, from the definition of t-exec() and of the condi- 
tional operation, q < a and for each finite prefix a' of a there is a timed execution a" 
of t-exec(QH) such that a' < a". It is sufficient to show that for each finite prefix a' 
of a there is a timed execution a" of t-exec(£lH q ) such that a' < (q ~ a''). Consider a 
prefix a' of a, and let a" be an element of t-exec(QH) such that a' < a". Then there is 
a time-enriched execution a'" of S7# such that a' < t-exec(a" r ), which means that there 
is a finite prefix a"" of a'" such that a' < t-exec(a'") and q < t-exec(a'"). Let g 8 - be 
the prefix of a"". We know that such prefix exists. Then, from the definition of H q , 
a""\>qi is a state of H q , and thus there is a time-enriched execution a' of S7# q such that 
a' < (g~ t-exec(a')). Moreover, t-exec(a') £ t-exec(Vu q ), which is sufficient to conclude. 
The argument can be reversed. 

Finally, we need to show that Pt- exec (p H )\c an( i Pt-exec(P H ) coincide on the cones of their sample 
spaces. Thus, consider a finite timed execution fragment a of M. From the definition of t-exec(), 

Pt-e X ec(V Hq )[Ca\= £ PH q [C q ,]. (9.31) 

q' £min(-{q , (E:states(Hq)\a<it-exec(q , )y) 



From (9.30), 



l^i\q'Estates(H>q t ) Ph[^ qi - 



Pt-exec(P Hq )[^a\ ~ 2^ V- p„\ C 1 • l 9 " 32 ) 

q'emin({q'estates(H q )\a<t-exec(q')}) ^ % Hl q ' S 

From the definition of the states of H q , (9.32) can be rewritten into 

D lr< 1 — 8 ^q'emin({q'estates(H>q t )\q^a<t-exec(q^q')}) ^Hl^q^q'} <„„„-, 

r t-exec(P Hq )[^a\ - T~P~ \CT~' i ' l^-OdJ 

By simplifying the concatenations we obtain 

D r r , -i 2-^q'emin({q'estates(H)\q^a<t-exec(q')}) H LWJ /„ „ ,1 

Pt-exec(V Hq )[Ca] - ^g] ■ l 9 " 34 ) 

From the definition of t-exec(), the definition of a conditional space, and the definition of the 
ft's, 



-Ti-esecCPuJIcJ^aJ - v^ n r^ 1 • (9.35J 



l^q'emin({q'estates(H)\q~a<t-exec(q')}) -TffL^i 

Since the right sides of Equations (9.34) and (9.35) are the same, we conclude that 

Pt-exec(P Hq )[Ca] = Pt-exec(P H )\C q [C q ~ a ]- (9.36) 

This completes the proof. ■ 

Conversely, we show that every probabilistic timed execution of M is sampled by some proba- 
bilistic time-enriched execution of M. Let if be a probabilistic timed execution of M. Then, 
build H' as follows. Let Hq be a probabilistic timed execution consisting of a single state that 
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is t-sampled by q$ , i.e., t-sample(q °) = q$ . Strictly speaking Hq is not a probabilistic timed 
execution because q ° should enable a transition in general. Suppose now that Hi is defined. 
Then build -ff;+i be extending the transition relation of Hi from all the states of Hi that do 
not end in 8 and do not have any outgoing transition as follows. Consider a state q of Hi that 
do not end in 8 and do not have any outgoing transition, and let q' be the state of H such 
that t-exec(q) = q' (our construction ensures that there is always such a state since q ends with 
a [0, 0]-trajectory). From the definition of a probabilistic timed execution fragment, there is 
a probabilistic time-enriched execution fragment H q * of M starting from Istate(q') such that 
V^ = truncate i state i q i\(t-exec(VH ,))■ Let H' , be obtained from H q i by removing all the tran- 
sitions from states where an action has occurred and by removing all the states that become 
unreachable. Then, extend Hi from q' with q' ~ H',, i.e., Hi + i>q' = H 1 ,. 

Then the states of H' are the union of the states of the H^s, the start state of H' is q °, 
and for each state q of H' , if q is a state of Hi, then £r;? = ^ r q * +1 ■ 

Proposition 9.3.9 t-sample(H') = H . 

Proof. We prove that Vh = t-exec{Vu')- Then the equality between t-sample(H') and 
H follows by induction after observing that t-sample(H') and H have the same start state 
and that for each state q, step q samp e ' = (q,truncate q (t-exec(VH')\C q )), and that step q = 
(q, truncate q (VH\C q )). 

For the sample spaces, consider an element a of 0#. Then, by definition of 0#, there is an 
execution ciocii • • • of H such that lim 8 - a,- = a, and such that either a is not a finite execution, 
or the last element of a ends in 8. We distinguish two cases. 

1. a is either an infinite sequence or a finite sequence a^ai ■ ■ ■ a n where a n ends with 8. 

From the definition of the transition relation of H', there is a sequence of extended time- 
enriched execution fragments qo,qi, ■ ■ ■ such that for each i a 8 - = t-exec(qo ~ • • • ~ g 8 ), 
Qo ~ <7i ~ ' ' ' is an element of 0///, and t-exec(qo " qi "■■■) = a. Thus, a £ £l t _ exec r H i\. The 
converse argument is a reversal of the argument above. 

2. a = (xqcxi ■ ■ -a n where a n ends with an open trajectory. 

From the definition of the transition relation of H', there is a sequence of extended 
time-enriched execution fragments qo, qi, . . . , q n -i such that for each i < n — 1 a,- = 
t-exec(qo ~ • • • ~ ft) and go ~ • • • ~ Qi is a state of H'. Furthermore, for each finite prefix 
a' of a there is a time-enriched execution fragment q n such that a' < t-exec(qo ~ • • -~ g n ) 
and <?o ~ • • • ~ <7n-i ~ <7n is an element of CIh 1 - This means that for each finite prefix a' of 
a there is an element a" of t-exec(£ljjt) such that a' < a", and thus a £ ^t-exec(V ,)• The 
argument can be reversed. 

Consider now a cone C a . From the definition of t-exec(), 

Pt-e X ec(Hi)[C ]= J2 P H'iC q }- (9-37) 

q(zmin({q(zstates(H r )\a<.t-exec(qyy) 

If C a is not empty, then a = a\ ■ ■ -a n , where a n = a, ato- • ■ a n _i is an execution of H , and 
there is a a' n such that a n < a' n and ai • • ■ a' n is an execution of H. We show by induction on 
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n that 

P H [C an ]= £ P H'[C q }- (9-38) 

q(zmin({q(zstates(H r )\a<.t-exec(qyy) 

The base case is trivial since C ao denotes the whole sample space. For the inductive case, from 
the definition of the probability of a cone, 

P H [C an ] = PHiCa^P^JCaJ. (9.39) 

From the definition of the transition relation of H , 

T>H \n 1 _ ^q£states(H')\t-exec(q) = a„- 1 *H' \S^ q\" t-exec(H'>q) \S^ a n >a n -\ \ . . 

^a„_ll^a„J- ^ p r r ] , (9.4UJ 

l^qestates(H')\t-exec(q) = a n - 1 r H'[^q\ 

where 

Pt-exec(H'>q)[Ca n >a n _ 1 ] = ^ P H'>q [C q>] ■ (9.41) 

q'(zmin({q'(zstates(H'>q)\a n <.t-exec(q^q'yy) 

Since a n -\ is a state of H, the last trajectory of a n -\ has domain [0,0], and the set {q £ 
states(H') \ t-exec(q) = a n -i} is a set of minimal states. Thus, by substituting (9.41) in (9.40), 
simplifying the numerator of (9.40), we obtain 



D r n -i _ ^^q'Emm(-[q'Estates(H')\a n <t-exec(q')}) ^ n L^gj /„ .„s 

r t-exec(H'>q)[^ a n >a n _ 1 J — ^ „ P77"j • {V.'iZ ) 



/-^q'Emin({q'Estates(H')\a n <t-exec(q')}) -Tff'L^^'J 
l^qEstates(H')\t-exec(q) = a n - 1 -Tff'L^^J 

By substituting (9.42) in (9.39), using induction and simplifying algebraically, we get (9.38 



Equivalent Probabilistic Time-Enriched Executions 

It is possible to define an equivalence relation on probabilistic time-enriched executions that 
captures exactly the probabilistic timed executions that they represent. 

Let Hi and Hi be two probabilistic time-enriched execution fragments of a probabilistic 
timed automaton M. Then t-execiVjjx) an d t-exec(VH 2 ) are sa id to be equivalent, denoted by 
t-execiVux) = t-exec(Vu 2 ), iff 

1. for each timed extended execution fragment a of M that does not contain infinitely many 
discrete actions, a G £L t -exec(V Hl ) iff « G ^t-exec(V H2 )'i 

2. for each finite timed extended execution fragment a of M, 

Pt-exec(V Hl )[Ca] = Pt-exec(V h 2 )[C a\- 

H\ and R^ are sa id to be equivalent, denoted by R\ = i?2, iff t-exec(q 1 ) = t-exec(q 2 ) and 
t-execiVjjx) = t-exec{Vu 2 )- 

Example 9.3.3 (Two equivalent probabilistic time-enriched executions) In the defi- 
nition above we do not require the sample spaces of the given probabilistic time-enriched ex- 
ecution fragments to contain the same timed executions with infinitely many discrete actions. 
Figure 9-2 shows an example of two probabilistic time-enriched executions whose corresponding 
sample spaces differ from a timed execution with infinitely many discrete actions and such that 
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Figure 9-2: Probabilistic time-enriched executions that represent the same probabilistic timed 
execution. 

t-sampleQ gives the same probabilistic timed execution. The important aspect of this example 
is that in the upper probabilistic time-enriched execution the explicit time-passage actions are 
used to let 1 time unit elapse in infinitely many different ways. However, the trajectory that 
is spanned before the first occurrence of action a is always the same. Observe that the fact 
that the two probabilistic time-enriched executions of Figure 9-2 represent the same structure 
is not a consequence of the limit closure of the sample space of t-exec(), since t-exec^Q,^) and 
t-exec(£lH 2 ) do not differ in timed executions that end with an open trajectory. Rather, by 
analyzing this example again in the context of parallel composition we will discover the reason 
for our definition of t-exec() (cf. Example 9.5.1). ■ 



The rest of this section is dedicated to showing that = characterizes the probabilistic timed 
executions represented by probabilistic time-enriched executions. We do it by showing two 
results: the first result says that two equivalent probabilistic time-enriched executions describe 
the same probabilistic timed execution, and the second result says that for each probabilistic 
time-enriched execution H , "P t _ samp i e tff\ = t-exec{Vjj)- 

Proposition 9.3.10 If t-exec(Hi) = t-exec(H2), then t-sample(Hi) = t-sample(H2). 

Proof. Let q £ states(t-sample(Hi)) . If q = t-exec(q 1 ) or q £ Qt-exedH-i) an( i contains finitely 
many discrete actions, then q £ states(t-sample(H2)) trivially. Thus, suppose that Itraj(q) is a 
[O,0]-trajectory and that there is a q' £ ^t-execlH!) such that q < q' . Then, Pt-execlHAi^'q] > 0? 
and, since t-exec(Hi) = t-exec^Hz), Pt-exec{H 2 )\Cq\ > 0- Thus, there is a q" £ $lt-exec(H 2 ) sucn 
that q < q", which means that q £ states(t-sample(H2)) ■ The converse argument is identical. 
Consider now a state q of t-sample(Hi) and t-sample(H2). We need to show that tr q sam ' p e( 1 > 

t-sample(H 2 ) 



q 
and tr q ""-"^'"y-" 2 ) are ^g same transition. From the definition of t-sample(), it is enough to show 
that truncate qit-execiVn^Cq) = truncate q (t-exec(VH 2 )\C q ). Since t-execiVjjx) = t-exec(VH 2 )i 
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a direct analysis of the definition of t-exec() shows that t-execiVjj^Cq = t-exec(VH 2 )\Cq- The 
truncation operation is independent of the elements of S7 that contains infinitely many discrete 
actions, and thus ^ltmncate q (t-exec(V Hl )\c q ) = ^tmncate q (t-exec(V H2 )\c q )- Furthermore, directly from 
the definition of =, P tr uncate q (t-exec(V Hl )\c q ) and P tr uncate q (t-exec(V H2 )\c q ) coincide on the cones, 
and thus truncate q^t-execiVjj^Cq) = truncate q (t-exec(VH 2 )\C q )- ■ 

Proposition 9.3.11 Let H be a probabilistic time- enriched execution of a probabilistic timed 
automaton M. Then, Vt_ samv i e m) — t-exec{Vjj)- 

Proof. Consider a finite timed execution a of M. We prove the proposition in three steps. 

1. For each finite timed extended execution a of M , there is a timed extended execution a' 
of & t- sample! H) such that a < a ' iff there is a timed extended execution a" of £lt-exec!V H ) 
such that a < a" . 

Let a' G ^t-sample(H) such that a < a' . Then there is a complete execution goft • • • of 
t-sample(H) such that lirrnqi = a'. In particular, there is a value n such that a < q n . 
From the definition of the transition relation of t-sample(H), Pt-exec!H)[C qn \ > 0, and thus 
there is a timed execution a" of £lt-exec!V H ) such that q n < a", which means that a < a". 
Conversely, suppose that there is a timed execution a" of Q, t _ exec t-p H \ such that a < a". If 
a" contains finitely many actions, then a" £ ^ t- sample! H) by definition. Otherwise, there 
is a finite prefix a'" of a" such that a < a'" and the last trajectory of a'" has domain 
[0,0]. From the definition of t-sample(H), a'" is a state of t-sample(H), and thus there 
is a timed execution a' of & t- sample! H) such that a'" < a', which means that a < a' . 

2. For each timed extended execution fragment a of M that does not contain infinitely many 
discrete actions, a G £L t -sam V le(H) iff a G &t-exec(V H )- 

Let a be a timed extended execution of M that does not contain infinitely many discrete 
actions, and suppose that a G ^t-sam-ple(H)- If a ends with 8, then Item 1 is sufficient 
to conclude that a G ^t-exec!V H )- If a does not end with 8, then there is a finite execu- 
tion goft • • 'Qn °f t-sample(H) such that q n ends with a right-open trajectory. From the 
definition of the transition relation of t-sample(H), q n G truncate qn _ 1 (t-exec(T ) H)\Cq n _ 1 ). 
Since q n ends with an open trajectory, q n G Sl t -exec(V H ), i- e -, « G &t-exec(V H )- 
Conversely, suppose that a G ^t-exec!V H )- If a ends with 8, then Item 1 is sufficient to 
conclude that a G ^t- sample! H)- If a does not end with 8, then there is a finite prefix a' of a 
such that a>a' does not contain any action, and either a' is the start state of t-sample(H), 
or the last trajectory of a' has domain [0,0]. Thus, from the definition of t-sampleQ, a' is 
a state of t-sample(H). From the definition of truncate, a G truncate a i(t-exec(V u)\C a i) , 
and thus, from the definition of the transition relation of t-sample(H), a G £tj amv e ^ ' . 
Since a ends with an open trajectory, a G 1 - sample! H)- 

3. For each finite timed extended execution fragment a of M , 

P t-sample(H)[^a\ = Pt-exec(V H ) [^ a\ ■ 

Let a be a finite timed execution. From Item 1, C a Samp = iff C a eX = 0- 

Suppose that C a Samp is not empty. Then there is an execution of t-sample(H), 
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ciocii • • -a n -ia n such that a n -\ < a < a n . From the definition of the probability of a 
cone, 

P t-sample(H)[^a\ = P, a [^ a t \P, a t [^ a 2 \ ' ' ' P. »„-2 L^a„_! J-*, 9„_! [^ a\ ■ (9.43) 

From the definition of t-sample(H), for each i < n 

P ai [Ca i + 1 ] = Pt-e X ec(H)\C ai [Ca i + 1 ]- (9-44) 

Thus, by substituting (9.44) in (9.43) and simplifying, we obtain 

Pt-sample(H)[C a ] = Pt-exec(H) [C a] • (9.45) 

This completes the proof. ■ 

9.4 Moves 

In the non-timed framework we have introduced the notion of a weak transition to abstract 
from internal computation. Informally, a weak transition is obtained by concatenating several 
internal and external transitions so that overall the system emulates a unique transition labeled 
with at most one external action. In the timed framework, due to the presence of explicit 
time-passage actions, it may be the case that some time t cannot elapse without performing 
some internal transitions in the middle. This problem becomes more evident when we extend 
the simulation relations to the timed framework (cf. Chapter 12). For this reason we introduce 
the concept of a move, which extends weak transitions and abstracts from internal transitions 
interleaved with time-passage transitions.. 

Let M is a probabilistic timed automaton, s be a state of M, V be a discrete probability 
distribution over states of M, and a be an action of M or the value 0. If a is a visible action of 
M then we use the expression s ~~» V to denote s =>■ V; if a = 0, then we use the expression 
s ~~» V to denote s ~~» V, which is the same as s =>■ V; if a is a time-passage action, i.e., 
a = d for some d G K + , then we use the expression s ~~» V to denote that V is reached from s 
by means of several internal and time-passage transitions so that in each situation time d has 
elapsed. Formally, s ~~» V iff there is a probabilistic execution fragment H such that 

1. the start state of H is s; 

2. Pfj[{a8 | aS £ 0^}] = 1, i.e., the probability of termination in H is 1; 

3. for each aS £ 0#, t-trace(a) = t-trace(a); 

4. V = lstate(8-strip(VH)), where 8-strip{Vjj) is the probability space V' such that 0' = 



{a \ aS E ^_ff}, and for each a £ 0', P'[a] = Ph[C, 



aS\ 



The notion of a generator for a weak transition can be extended to moves in a straightforward 
way. 
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9.5 Parallel Composition 

The parallel composition operator for probabilistic timed automata is exactly the same as the 
parallel composition operator for probabilistic automata. Thus, we omit the formal definition. 
According to the definition of the transition relation of M1HM2, Mi and Mi synchronize on 
all their time-passage transitions, and thus time advances always at the same speed in M\ and 
M 2 . 

The definition of a projection of a probabilistic time-enriched execution is the same as the 
definition of a projection of a probabilistic execution, except that the states of a probabilistic 
time-enriched execution fragment are time-enriched execution fragments rather than ordinary 
execution fragments. Thus, we need to extend the definition of a projection to time-enriched 
execution fragments and time-enriched transitions. 

Let M be M1HM2, and let a be a time-enriched execution of M. The projection of a onto 
Mi, i = 1,2, is the sequence obtained from a by projecting the codomain of each trajectory 
onto Mi, by removing all the actions not in acts(Mi), and by concatenating all the trajectories 
whose intermediate actions are removed. It is straightforward to check that a is a time-enriched 
execution of M 8 -. 

Let if be a probabilistic time-enriched execution of M, and let tr = (q,V) be an action 
restricted transition of H such that only actions of Mi, i = 1,2, appear in tr. Define the 
projection operator on the elements of S7 as follows: (a,q')\Mi = (a,q'\Mi), and 8\Mi = S. 
The projection of tr onto Mi, denoted by tr\Mi, is the pair (q\Mi,V\Mi). 

Proposition 9.5.1 Let M = M\\\M2, and let H be a probabilistic time- enriched execution 
fragment of M . Then H\M\ £ t-prexec(M\) and H\Mi £ t-prexec(Mi) ■ 

Proof. The structure of the proof is the same as the proof of Proposition 4.3.4. This time it is 
necessary to observe that for each state q of H the transition (tr^, \ acts(Mij) \M\ is generated 
by a time-enriched transition of M 8 -. ■ 

Proposition 9.5.2 Let M = M\\\M2, and let H be a probabilistic time-enriched execution 
fragment of M . Let Hi be H\Mi, i = 1,2. Let q be a state of Hi. Then, 

P Ht [C q ]= £ P H [C q .]. (9.46) 

q' (zmin(q~\H) 

Proof. This proof has the same structure as the proof of Proposition 4.3.5. ■ 

In the rest of this section we extend the results of Section 9.3.3 to account for parallel com- 
position. We show that sample commutes with projections and that the projections of two 
equivalent probabilistic time-enriched executions are equivalent. The first result guarantees 
that sample and projection are well defined for probabilistic time-enriched executions; the sec- 
ond result allows us to define indirectly a projection operator on probabilistic timed executions: 
namely, given a probabilistic timed execution H of Mi||M2, let H' be any probabilistic time- 
enriched execution of Mi||M2 such that t-sample(H') = H. Then, H\Mi is defined to be 
t-sample(H'\Mi). Before proving these two results, we show why in the definition of t-exec() 
we force probabilistic time-enriched executions like those of Figure 9-1 to be mapped to the 
same structure (cf. Example 9.3.2). 
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Example 9.5.1 (Reason for the definition of t-exec) We have already seen that the prob- 
abilistic time-enriched executions of Figure 9-2 are t-samples of the same probabilistic timed 
execution. Suppose now the probabilistic time-enriched executions of Figure 9-2 to be proba- 
bilistic time-enriched executions of the parallel composition of two probabilistic timed automata 
Mi and M2, and suppose that a is an action of Mi only. By projecting the probabilistic time- 
enriched executions of Figure 9-2 onto M\ we obtain two probabilistic time-enriched executions 
like those of Figure 9-1, which must denote the same probabilistic timed execution if we want 
t-sample to be preserved by the projection operation. ■ 

Proposition 9.5.3 Let M be M\\\M2, and let H be a probabilistic time- enriched execution of 
M. Then, sample(H\Mi) = sample(H)\Mi. 

Proof. Since the sampling function commutes with the projection function, sample(H\Mi) 
and sample(H)\Mi have the same states. 

For convenience, denote sample(H) by H'. Let q be one of the states of sample(H)\Mi. 
Below we show that the equation for the transition leaving from q in sample(H)\Mi and the 
equation for the transition leaving from q in sample(H\Mi) denote the same transition. This 
is sufficient to show that sample(H)\Mi and sample(H\Mi) have the same transition relation. 
We use implicitly the fact that the projection onto M 8 - distributes over the sum of transitions 
restricted to acts(Mi). 

From (9.25), Proposition 4.3.2, and an algebraic simplification, the expression 

E f} H 'Pf[acts{Mi)\{trf \acU{Mi))\Mi (9.47) 

q'£q\H> 

can be rewritten into 

E E fJ H 'p;r le ~ 1{9,) sample(tr^ \ actsiM^M^ (9.48) 

q'eq~\H' q"esam-ple~ 1 (q') 

which becomes 

E p?Z/e( g »)^ r "' e " 1(sam " e(9 " ))sara ^«' r "ctemw, (9.49) 

q"esam-ple~ 1 (q]H r ) 

after grouping the two sums. 

Denote H\Mi by H" . From (4.22), Proposition 4.3.2, and an algebraic simplification, 

E p s q r p ' e ~ 1{q) sample(trf) (9.50) 

q'(zsample~ (g) 

can be rewritten into 

E E P^ mVle ~ 1(q) p q ^ H P^acts{M i )]sample{tr^, \ acts{Mi))\Mi, (9.51) 

q'esample~ 1 (q) q"eq'~\H 

which becomes 

E p;^ e " 1(9) p;f rM ° lff P^[acfo(M 8 )] S amp/ e (^ \ acts(Mi))\Mi (9.52) 

q" Example -1 (q))]H 
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after grouping the two sums. 

From the commutativity of sample and projection, sample -1 (q\H r ) = sample -1 ( q)~\ H . 
Thus, in order to show that (9.49) and (9.52) denote the same transition, it is sufficient to 
show that for each state q" of sample -1 (q~\H r ) , 

ffl\ H ' sample' 1 (sample(q")) _ sample' 1 (q) ( q"\M,)]H /„ rn\ 

I J sample(q"y J q" _ *V [M; fq" ' (V-O-l) 

By expanding the expressions above with their definitions, (9.53) becomes 

PH'[C ' S gmple(q")]PH[C ' q ii] 

(J2q>emin(q~}H r ) P H'[Cq'])(J2q" esample' 1 {sample{q")) P h[C q"}) 

_ PH"[C 'q"[M,]PH[C 'g"] 

(J2q' es ample- 1 (q) P H" [ C q'])(J2q" Emm{{q" \M t )]H) P h[C q'']) 

By simplifying common subexpressions, using Proposition 4.3.5, and observing that 

Pw[C samv i e ( q n)] = E PH[Cq»], (9.55) 

q" ^sample (sample(q ff )) 

(we have verified properties like (9.55) several times) Equation (9.54) becomes 

E Pw[Cq,}= J2 Ph»[C?], (9.56) 

q'Emin(q]H') q'Esample -1 (q) 

which can be shown as follows: 

E P H'lCq>] 

q' £min(q~\H') 

E E ph[c 9 »] 

q'Emin(q~\H') q" Esample -1 (q 1 ) 

E P h[CA 

q" £min(s ample (q~\H f )) 

E P HlCq"} 

q" £min ((sample - (q))~\H) 

E E phVA 

q'Esample 1 (q) q" Emin(q'~\H) 

E Ph»[C?], 

q'Esample~ (g) 

where the first step follows from (9.55), the second and fourth steps follow from grouping and 
ungrouping sums, the third step follows from the commutativity of sample and projection, and 
the fifth step follows from Proposition 4.3.5. ■ 

Proposition 9.5.4 Let Hi and Hi be two probabilistic time-enriched executions of M\\\Mi- If 
#1 = H 2 , then H t \M t = H 2 \M t , i = 1,2. 
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Proof. We show first that t-exec(T > H 1 rM t ) an d i- exec (T y H 2 \M i ) assign the same probabilities 
to the same cones; then we show that the sample spaces of t-exec(T > H 1 rM t ) an d i- exec (T y H 2 \M i ) 
satisfy the condition for =. This part of the proof relies on the way we have defined the sample 
spaces of the objects produced by t-exec(). For the cones, we show that for each finite timed 
extended execution a of M 8 -, 

P t-exec(V HllMi )[Ca]= \^ Pt-exec^) [C a >] ■ (9.57) 

a'Emin({a'Et-frag*(M 1 \\M 2 )\a = a'\M t }) 

and 

P t-exec(V H2[M .)[Ca] = /2 Pt-exec(H 2 )[C '«']• (9.58) 

a'emin({a'et-frag^(M 1 \\M 2 )\a = a'\Mi}) 

Then, since Hi = H2, we conclude that the right sides of (9.57) and (9.58) are equal, and thus, 
H\\Mi = H%\Mi. We prove only (9.57); the proof for (9.58) is symmetric. From the definition 
of t-exec(), 

P t-exec(V Hl{M .)[Ca]= £ Ph^M^^. (9.59) 

q£min(-{q(E:states(Hi \ Mi)\a <it- exec (<?)}) 

From (4.31), 

Pt-exec(V HllMt )[Ca]= £ I £ P ^ [<?,'] I • (9-60) 

q(zTtiin(-[q(zstates(Hi\M l )\a<.t-exec(q)}) \q' (zmin(q~\Hi) J 

Consider a state q of min({q £ states(H\ \Mi) \ a < t-exec(q)}) and a state q' of min(q\Hi). 
Then, from the definition of t-exec(), there is at least one a' £ t-frag* s (Mi\\M2) such that 
a = a!\Mi and q' £ min({q' £ states(Hi) \ a' < t-exec(q')}). Moreover, there is exactly 
one minimum a'. Conversely, consider one a' £ min({a' £ t-frag* s (Mi\\M2) \ a = a'[M 8 }), 
and consider a state q' of min({q' £ states(Hi) \ a' < t-exec(q')}). Let q = q'\Mi. Then, 
q' £ min(q\Hi) and q is a state of min({q £ states(Hi \M{) \ a < t-exec(q)}). Thus, from (9.60) 
we obtain (9.57). 

We now move to the sample spaces. Let a be an element of Qt- exec (v H [M .) that does not 
contain infinitely many discrete actions. If a ends with 8, then a is trivially an element of 
^t-exec(T H2[Mi ) since P t-exec(V H2[M .)[C a ] = p t-exec(V H2 Wt ) [C a ] > 0. Otherwise, a ends with an 
open trajectory. Then, from the definition of Qt- exec (v H rM .)? for each finite prefix a' of a there 
is an element a\ of t-exec(^lff 1 rjif t ) such that a' < a\. It is enough to show that for each finite 
prefix a' of a there is also an element a.^ of t-execi^u^j^.) such that a! < a.^- 

Let a' be a finite prefix of a such that there is an element a\ of t-exec(£l Hl r Mt ) such that 
a' < a\. Thus, there is a time-enriched execution a[ of Slji^Mi such that a 1 < t-exec^a'-^). 
This means that there is a state q\ of H\\Mi such that a' < t-exec(qi). From the definition 
of projection, there is a state q[ of Hi such that a' < t-exec(q[ |~M 8 ), and thus there is a timed 
execution a'{ of t-exec(£ln^) such that a' < (a"[M 8 ). Consider a finite prefix a"' of a'{ such 
that a' < (a'{'\Mi). Then, P t -exec(V Hl )[C a '('] > 0. Since H x = H 2 , Pt-exec(V H2 )[C a '('] > 0, which 
means that there is a timed execution a" of Qt- exec (v H ) such that a' < (a" \Mi). Thus, there 
is a state q' 2 of H 2 such that a' < t-exec(q' 2 \Mi), and from the definition of projection, there 
is a state q2 of H2 \Mi such that a 1 < t-exec(q2). This implies that there is an element a' 2 of 
t-exec(^l H2 r Mt ) such that a' < a' 2 , which is sufficient to conclude. ■ 
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9.6 Discussion 

To our knowledge, no general probabilistic models with dense time have been proposed except 
for the automata of Courcoubetis, Alur and Dill [ACD91a, ACD91b]. In our model no prob- 
ability distributions over passage of time are allowed within a probabilistic timed automaton; 
time can elapse probabilistically only within a probabilistic timed execution, and the associated 
probability distributions can be only discrete. We have chosen to define the timed model with 
such a restriction so that all the theory for the untimed model carries over. 

Further work should investigate on the extension of our model to non-discrete probability 
distributions. A starting point could be the study of restricted forms of non-discrete distri- 
butions as it is done by Courcoubetis, Alur and Dill in [ACD91a, ACD91b]. Useful ideas can 
come from the work on stochastic process algebras of Gotz, Herzog and Rettelbach [GHR93], 
Hillston [Hil94], and Bernardo, Donatiello and Gorrieri [BDG94]. 
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Chapter 10 

Direct Verication Time 
Complexity 



Part of this chapter is based on joint work with Anna Pogosyants and Isaac Saias; some of the 
ideas have been influenced by discussion with Lenore Zuck. The verification of the randomized 
dining philosophers algorithm of Lehmann and Rabin (Section 10.6) is based on joint work 
with Nancy Lynch and Isaac Saias [LSS94]; the verification of the randomized algorithm for 
agreement of Ben-Or (Section 10.8) is joint work with Anna Pogosyants and is a formalization 
of a proof that appears in the book on distributed algorithms of Nancy Lynch [Lyn95]. Close 
interaction with Anna Pogosyants lead us to the idea of the abstract complexity measures of 
Section 10.7. 

10.1 General Considerations About Time 

The direct analysis of a probabilistic timed automaton is carried out exactly in the same way 
as for untimed probabilistic automata. Thus, probabilistic statements and progress statements 
can be generalized directly, and the coin lemmas can be applied without any modification. 

In this chapter we concentrate more on topics that are specific to the presence of time. In 
particular, it is now possible to enrich the notation for progress statements and verify some of 
the real-time properties of a probabilistic timed automaton. We extend the progress statements 
of Chapter 5 by adding a time parameter t: the expression U — ► U' means that, starting from 

a state of U, a state of U' is reached within time t with probability at least p. Based on the new 
timed progress statements we show how to derive upper bounds on the worst expected time for 
progress. 

We generalize the method for time complexity analysis to more abstract complexity mea- 
sures. Then, rather than studying the expected time for progress, we study the expected 
abstract complexity for progress. We use abstract complexity to derive an upper bound on the 
worst expected time for decision of the randomized algorithm for agreement of Ben-Or that we 
presented in Chapter 5. Specifically, we show that under some conditions on the scheduling 
policy, each non-faulty process completes its i stage within some upper bound, and we show 
an upper bound on the expected number of stages that are necessary to reach agreement. In 
this case the abstract complexity is the number of stages. A direct analysis of the expected time 
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for success in Ben-Or's algorithm would not be as easy since there is no useful upper bound on 
the time it takes to a process to move from a stage to the next stage. 

Sections 10.2, 10.3, and 10.4 simply extend the definitions of Chapter 5 to the timed case; 
Section 10.5 shows how to derive upper bounds on the worst expected time for progress given 
a timed progress statement, and Section 10.7 shows how to derive upper bounds on the worst 
expected abstract complexity for progress given a timed progress statement with abstract com- 
plexity; Sections 10.6 and 10.8 present examples of application by proving that the randomized 
dining philosophers algorithm of Lehmann and Rabin guarantees progress in expected constant 
time and that the randomized agreement algorithm of Ben-Or guarantees agreement in expected 
exponential time. 

10.2 Adversaries 

An adversary for a probabilistic timed automaton M is a function A that takes a finite timed 
execution fragment a of M and returns a timed transition of M that leaves from Istate(a). 
Formally, 

A : t-frag*(M) -+ t-trans(M) 

such that if A(a) = (s,V), then s = Istate(a). Moreover, an adversary satisfies the following 
consistency condition: if A(a) = (s,V), then for each prefix a' of some element a" of 0, 
A(a^ a') = (Istate(a') ,V>ot') . Informally, consistency says that an adversary does not change 
its mind during a timed transition. 

An adversary is deterministic if it returns either deterministic timed transitions of M or 
pairs of the form (s,V(s8)), i.e., the next timed transition is chosen deterministically. Denote 
the set of adversaries and deterministic adversaries for a probabilistic timed automaton M by 
Advs(M) and DAdvs(M), respectively. 

The definitions of an adversary schema and of the result of the interaction between an adver- 
sary and a probabilistic timed automaton is the same as for the untimed case (cf. Section 5.2), 
and thus we do not repeat them here. 

To guarantee that our adversaries are well defined, we need to prove the following lemma. 

Lemma 10.2.1 If (s,V) is a timed transition of a probabilistic timed automaton M , then for 
each prefix a' of some element a" of CI, (lstate(a'),V>a r ) is a timed transition of M . 

Proof. This is proved already in Proposition 9.3.5. ■ 

10.3 Event Schemas 

As for the untimed case we need a mechanism to associate an event with each probabilistic 
timed execution fragment of a probabilistic timed automaton. Thus, an event schema is a 
function e that associates an event of the space Vh with each probabilistic timed execution 
fragment H of M. The notion of finite satisfiability extends directly from the untimed case. 
Observe that, although in Vh there can be uncountably many cones, each finitely satisfiable 
event can be expressed as the union of countably many disjoint cones. Furthermore, every 
uncountable family of cones contains at least two cones that are not disjoint. 
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The definition of a timed probabilistic statement extends directly from the untimed case, and 
similarly the definition of the concatenation of two event schemas extends directly. Therefore, 
we omit the definitions, which are identical to those of Chapter 5. 

Proposition 10.3.1 The concatenation of two event schemas is an event schema. That is, if 
e = e\ o Cones e 2> then e is an event schema. 

Proof. Consider a probabilistic timed execution fragment H . From Proposition 9.3.3 each set 
e 2{H\q) is an event of Tu- From the closure of a u-field under countable union, e(H) is an 
event of Tu- ■ 

Proposition 10.3.2 P H \e x o Cones e 2 (H)] = J2 qe c'ones(H) PH[C q ]P H \ q [e 2 (H\q)]. 

Proof. Since Cones(H) represents a collection of disjoint cones, from (5.13) we obtain 

P H [ei ocones e 2 (H)]= J2 PH[e 2 (S\q)]. (10.1) 

gG Cones(H) 

From Proposition 9.3.3, for each q £ Cones(H) 

P H [e 2 {H\q)\ = P H [C q }P Hlq [e 2 (H\q)}. (10.2) 

By substituting (10.2) in (10.1) we obtain the desired result. ■ 

Now it is possible to prove a concatenation property similar to the one for the untimed case. 
Proposition 10.3.3 Consider a probabilistic timed automaton M . Let 
1- ??Advs,e(ei) TlV\ and , 

2. for each A G AdvS, q£®, let ^^Advs,Cones{ V re X ec{M,A, q )){ e 2) K P2- 

Then, Pr AdvS)@ (ei o Co nes e 2 ) TlpiP2- 

Proof. Consider an adversary A G Advs and any finite timed execution fragment q G 0. Let 
H = prexec(M,A,q). From Proposition 10.3.2, 

P H [ei ocones e 2 (H)}= J2 PH[C q >]P HW [e 2 (H\q')]. (10.3) 

q 'eCones(H) 

Consider an element q' of Cones(H). It is a simple inductive argument to show that 

H\q' = prexec(M,A,q'), (10.4) 

where we use consistency for the base case. Thus, from our second hypothesis, 

P Hlq ,[e 2 (H\q')]np 2 . (10.5) 

By substituting (10.5) in (10.3), we obtain 

P H [ei ocones e 2 (H)]TZ P2 ]T P H [C q ,]. (10.6) 

q 'eCones(e 1 (H)) 
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By using the fact that Cones(H) is a characterization of ei(H) as a disjoint union of cones, 
Equation (10.6) can be rewritten into 

P H [ei ocones e 2 (H)}n P2 P H [e 1 (H)}. (10.7) 
From the first hypothesis, Pu\ei(H)\ 1Z p\; therefore, from Proposition 5.4.1, 

P H [eio C ones e 2 (H)]n Pl p 2 . (10.8) 

This completes the proof. ■ 

10.4 Timed Progress Statements 

As a special case of a probabilistic statement for the timed case we can add some features 
to the notation X — >Advs X'. In particular we define a timed progress statement to assert 

that starting from a set of states U some other state of a set U' is reached within time t with 
probability at least p. Such a statement, which we denote by U — >Advs U' , or by U — ► U' if 

Advs is clear from the context, is expressed by the probabilistic statement ~PiAdvs,u( e U',t) > P, 
where the event schema ejji t applied to a timed probabilistic execution fragment H returns the 
set of timed executions a of £ljj where a state from U' is reached within time t in a>qQ . Such 
a set can be expressed as a union of cones, and therefore it is an event. 

Similarly, the progress statements involving actions can be generalized to the timed frame- 
work. Thus, V — >Advs V is the probabilistic statement Pr Advs ,& v v i( e V ,t) > Vi where ®v,V 1S 
the set of finite timed execution fragments of M where an action from V occurs and no action 
from V' occurs after the last occurrence of an action from V, and the event schema ey\t applied 
to a timed probabilistic execution fragment H returns the set of timed executions a of Q,jj such 
that an action from V occurs in a>qQ within time t. 

In order to generalize the concatenation theorem for progress statements, we need to extend 
the definition of a finite-history-insensitive adversary schema. Thus, an adversary schema Advs 
is finite-history-insensitive iff for each adversary A of Advs and each finite timed execution 
fragment a of M there is an adversary A' of Advs such that for each timed execution fragment 
a 1 such that a < a' , A(a') = A'(a'\>a). Then, the following theorem is shown in the same way 
as for the untimed case. 

Theorem 10.4.1 Let Advs be finite-history-insensitive. If X —^Advs X 1 and X' —^Advs X", 
then X - — ?Advs X". ■ 

Pl'P2 

10.5 Time Complexity 

In this section we show how to study the time complexity of a randomized distributed algorithm. 
We start by defining how to compute a worst expected time, and then we show how it is possible 
to derive upper bounds on the worst expected running time of an algorithm based on timed 
progress statements. 
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10.5.1 Expected Time of Success 

Let e be a finitely satisfiable event schema and suppose that Pfj[e(II)] = 1, i.e., that the property 
described by e is satisfied in H with probability 1. Let Cones(H) be a characterization of e(H) 
as a disjoint union of cones, where each element of Cones(H) identifies the first point along 
a timed execution where the property denoted by e is satisfied. Then, we can compute the 
expected time to satisfy the property identified by e as 

J2 P H [C q ](ltime(q>qX)). (10.9) 

gG Cones(H) 

In general, if e is a finitely satisfiable event-schema and Cones(H) identifies the first point along 
a timed execution where the property identified by e is satisfied, then for each probabilistic timed 
execution fragment H of M we define _Ejj[e], the expected time to satisfy e in if , as follows. 

E r n = f J2 q eCones(H) P H[C q ](ltime(q>qV)) if P H [e(H)] = 1 
1 oo otherwise. 

Then, the question is the following: are there easy ways to compute upper bounds on the 
expected time for success in a randomized algorithm without computing explicitly (10.10)? We 
give a positive answer to this question. 

10.5.2 From Timed Progress Statements to Expected Times 

Timed progress statements can be used to analyze the time complexity of a randomized algo- 
rithm. The main idea for the analysis is expressed by Proposition 10.5.1. Suppose that we 
know the following: 

| U -U A dvs U' 

{ U => (U Unless U'). 

Then, if Advs is finite-history-insensitive and s8 ^ &Ms) f° r eac h A G Advs and each s G U, 
we know from Proposition 5.5.6 that U — >Advs U' . Let e be a finitely satisfiable event schema, 
and let Cones express the points of satisfaction of e. Suppose that for each probabilistic timed 
execution fragment H and each state q of H , if there is no prefix q' of q such that q' G Cones(H), 
then e(H>q) = e(H)>q and Cones(H>q) = Cones(H)>q (e.g., e can express the property of 
reaching some state in a set U", or the property of performing some action). Let 

Eu,Advs[ e ] = su PseU,AeAdvs E prexec(M,A,s)[ e ]- (10.12) 

Then the following property is valid. 
Proposition 10.5.1 

Eu,Advs[e] < t + pEu> jA dvs[e] + (1 - p)Eu,Advs[e\- (10.13) 

Proof. We prove (10.13) by distinguishing four cases. 

1- Ejj^ Advs [e] > Eu,Advs[e\- 

In this case (10.13) is satisfied trivially. 
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2. E UtAdvs [e] = oo and p < 1. 

Also in this case (10.13) is satisfied trivially. 

3. E UtAdvs [e] = oo and p = 1. 

We show that Ejj' t Advs [ e ] = °°7 which is enough to satisfy (10.13). Suppose by contradic- 
tion that Ejji Ad vs [e] < oo. Then we distinguish the following cases. 

(a) There is an adversary A of Advs and a state s of U such that 

P r re X ec(M,A, S )[ e (P reXeC ( M ^^ S ))] < 1 - 

(b) It is not the case that there is an adversary A of Advs and a state s of U such that 
P P rexec(M,A,s)[e(prexec(M,A,s))] < 1. 

For Case (a), let Conesjj' be the function that expresses the points of satisfaction of ejji, 
and let H be prexec(M, A, s), where P vrexec i M ^Ae(prexec(M,A,s))] < 1. Then, 

Pff[eGff)]> X) P H [C q ]P H >M H >9)), (10.14) 

gG ConeSjji(H) 

i.e., the probability of satisfying e is not smaller than the probability of reaching U' and 
then from there satisfying e. From the finite-history-insensitivity of Advs, for each state q 
of Conesijt(H) there is an adversary A 1 of Advs such that ift>g = prexec(M, A', Istate(q)), 
and thus, since Eu> ^ vs \e\ < oo, PH >q (e(H\>q)) = 1. By substituting this result in (10.14), 
we get 

P H [e(H)}> J2 P H[C q ]- ( 10 - 15 ) 

gG ConeSjji(H) 

Since p = 1, the right side of (10.15) is equal to 1, i.e., Pfj[e(H)] > 1, a contradiction. 

For Case (b), let Conesjji be a function that expresses the points of satisfaction of ejj, 
and, for each d > 0, let Cones,i be a function that expresses the event of reaching time 
d as a union of disjoint cones. From the definition of a probabilistic timed execution, 
we know that Cones,i exists and that for each probabilistic timed execution fragment H 
and each q £ Conesd(H), ltime(q>qQ) = d. Let H be prexec(M,A,s). From (10.10) the 
expected time for success for e is 

E H [e]= J2 P H [C q ]ltime(q>qX). (10.16) 

gG Cones(H) 

Let e be an arbitrary positive number. Let ©i be the set of elements q of Conesjji(H) 
such that ltime(q>qQ) < i + e, and let i?2 be the set of elements q of Cones t+e (H) that do 
not have any prefix in ©i. Since Pff[e[/(if)] = 1, then Pff[U ge e 1 ue 2 Cg] = 1- Moreover, 
by hypothesis, -Pff[U ge cw es (mC g ] = 1. Thus, observe that each element of Cones(H) has 
either a proper prefix or a suffix in ©i U ©2- In fact, if there is an element q of Cones(H) 
that has no prefix nor suffix in ©iU©2, then the cone C q would not be part of U q £e 1 ue 2 Cq> 
contradicting the hypothesis that -Pff[U ge cw es (mC g ] = 1. Similarly, we can show that 
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for each element q of ©i U ©2 has either a prefix or a proper suffix in Cones(H). Thus, 
Cones(H) can be partitioned into two sets Q' p and s of elements that have a proper 
prefix and a suffix, respectively, in ©i U ©2, and ©i U ©2 can be partitioned into two sets 
©^ 2 and ©^ 2 of elements that have a prefix and a proper suffix, respectively, in Cones(H). 
Based on these observations, the right side of Equation( f0.f6) can be rewritten into 

E E PH[C q ,}PH»AC q>q ,}(ltime(q'»qX) + ltime(q»q'))\ (10.17) 

^qeep q 'ee s 12 \g'<g 

J2 J2 PH[C q ]PH> q [C q i >q ]ltime(q»ql?) . 

^eev e e[ j2 |g< g ' / 

Observe that for each q £ s , J2 q 'e& p \q<q' PH> q [C q ' >q ] = 1, and observe that for each 
l' £ ©1,25 Y, q e&p\q'<q PH> q '[C q>q i] = 1. By exchanging the sums in (f0.f7) and using some 
simple algebraic manipulations, we obtain 

E P H[C q '} lltime(q'»qX)+ ]T P H>q ,[C q>q ,]ltime(q>q') ] | (10.18) 

yeef 2 \ q eep\ q '< q 

E E PH[C q ]PH> q [C q > >q ]ltime(q>qQ) 
^'eef 2 gee«| g <g' 

In the first summand, since from the properties of e for each q' £ ©12? e(H\>q r ) = 
e(H)\>q', the subexpression J2 q e& p \q'<q ltime(q>q')Pjj >q i[C ' q>q i] denotes Ejj >q i[e]. In the 
second summand, observe that for each q' £ ©^ 2 there is exactly one element q of s 
such that q < q' . Moreover, Ph[C q ]PH> q [C q i >q \ = -Pff[C g /]. Thus, from (10.18) we obtain 

E H [e] < J2 PH[C q i](ltime(q'»qg) + E H>q ,[e])\ (10.19) 

\q'e@' li2 

J2 P H [C ql ]ltime(q'>qX) 

By repartitioning 0j 2 U0j 2 into ©i and ©2, and by observing that for each element q 
of ©i ltime(q>qQ) < t + e, and for each element q of ©2 ltime(q>qQ) = t + e, (10.19) can 
be rewritten into 

E H [e]<(t + e)l J2 PH[C q ]E H>q [e])\ + \ ]T P H [C q ]E H>q [e] | (10.20) 
^eej^n©! J y qe& p h2 ne 1 

J2 PH[C q ]E H>q [e] + J2 PH[C q ]E U)Advs [e] | , 

v gee| 2 ne 2 J \qee p 2 n& 2 
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where we have added Eu >q [e] in the upper right summand and Ejj tAdvs [e] i n the lower 
right summand. Since Advs is finite history insensitive, for each q £ ©i U ©2 there is an 
adversary A' of Advs such that (H>q) = prexec(M,A,lstate(q)). Thus, (10.20) can be 
rewritten into 

E H [e] <(t + e)l^2 P H [C q ]Eu, tAdvs [e]) I + ( E PH[C q ]E UtAdvs [e] ] , (10.21) 

\5fe0i / \qee 2 J 

where we have used U =>■ (U Unless U') to say that the last states of the elements of ©2 
are in U . Observe that J2 q e& PhICq] is P£f[ejy/ jt (i7)], which is 1 by hypothesis. Since by 
hypothesis Ejj^ Advs [e] < °°? from (10.21) we derive that Ejj tAdvs [e] < °°? a contradiction. 

4. E U}Advs [e] < oo, Eu> tAdvs [e] < oo, and Eu> tAdvs [e] < Eu> tAdvs [e\. 

Let A be an adversary of Advs and s be a state of U. Let H be prexec(M,A,s). Let e 
be any positive real number. Equation (10.21) can be derived also in this case using the 
same identical argument as before. Since we have assumed that Ejj^ Advs [e] < Ejj tAdvs [e], 
the lowest possible value of the right side of (10.21) occurs by giving U' the lowest possible 
probability, which is p. Thus, (10.21) becomes 

E H [e] <(t + e)pE v , Advs [e\ + (1 - p)E UAdvs [e\. (10.22) 

Since Equation (10.22) is valid for any adversary Advs and any state of U, we obtain 
timed execution fragment 

Eu,Advs[e] <(t + e)pE uljAdvs [e] + (1 - p)E U)Advs [e]. (10.23) 

Since Equation (10.23) is valid for every e, Equation (10.23) is valid also for the infimum 
of the values that e can have, i.e., 0, and thus, 

Eu,Advs[e] < t + pEu, jAdvs [e] + (1 - p)E U)Advs [e]. (10.24) 

This completes the proof. ■ 

Example 10.5.1 (From timed progress to expected time) As a simple example of ap- 
plication of Proposition 10.5.1, suppose that e expresses the property of reaching U'. Then, we 
know by definition that Ejj^ Advs [e] = 0. By applying Equation (10.13), we obtain Ejj tAdvs [e] < 
t + (1 — p)Ejj tAdvs [e], which gives Ejj tAdvs [e] < t/p, i.e., the expected time to reach U' from U 
is at most t/p. Informally speaking, we can view the process of reaching U' as a sequence of 
Bernoulli trials, each one performed every t time units. At time t, with probability p we have 
reached U' , and with probability (1 — p) we are still in U, and thus we apply the same exper- 
iment again. The expected number of rounds of such a process is 1/p, and thus the expected 
time for success is t/p. Suppose now that we know the following, 

{Uo -^Advs U\ U => (U Unless Z7i) 
f (10.25) 

Ui -^Adv S U 2 Ut^iUt Unless U 2 ), 
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and suppose that e expresses the property of reaching U 2 . Then, we know that Eu 2 ,Advs\. e \ = 0. 
By applying Proposition 10.5.1, we obtain 



| Eu ,AdvA e ] < h + PiEu lt Advs[e\ + (1 - Pi)Eu ,Advs[e] 
\ E Uu Advs[e] < h + (1 - P2)E Ul ,Adv.s[e]- 

From simple algebraic manipulations (10.26) becomes 

I Eu 0) Advs[ e ] < h/pi + Ejji^dvsie] 
\ E Uu Advs[e] < h/p 2 , 

and thus, after substituting the second inequality in the first inequality, 



'10.26) 



'10.27) 



'10.28) 



f E Uot Advs[e] < hi Pi + h/P2 
\ E Uu Adv.s[e] < h/P2- 

Suppose now that in addition to (10.25) we know that 

f U -^Ad.s U 2 

[ U ^(U Unless U 2 ), 
which is possible if U\ C Uq U U 2 - Then, from Proposition 10.5.1 we get 

E Uo ,Adv,[e]<t 3 /p 3 , (10.30) 

which added to (10.28) gives 



f E Uot Advs[e] < min(t 1 /p 1 + t 2 /p 2 ,t 3 /p 3 ) 
\ E Uu Adv.s[e] < h/P2- 

Therefore, more information may give us the possibility to prove better bounds. 



'10.31) 



Proposition 10.5.1 can be proved also for timed progress statements that involve sets of actions 
rather than sets of states. Let V,V denote two sets of actions, and let Advs be an adversary 
schema. Suppose that 

V -U A dv, V. (10.32) 

Let e be a finitely satisfiable event schema, and let Cones express the points of satisfaction of 
e. Suppose that for each probabilistic timed execution fragment H and each state q of H , if 
there is no prefix q' of q such that q' £ Cones(H), then e(H>q) = e(H)>q and Cones(H>q) = 
Cones(H)>q. Let E v ,v',Advs[e] denote sup qeQvvl)AeAdvs E prexec ^ M}A ^[e]. Let Q v , denote the 
set of finite execution fragments of M whose last action is in V', and let Ev> Advs[ e ] denote 
su Pgee v ,,AeAdv S E p rexec{M,A, q )[ e ]- Suppose that q'S £ VL A{q) for each q', each A G Advs and each 
q G ®v,V- Then the following proposition is valid. 

Proposition 10.5.2 

1- E v ,v> ,Advs[e] < t+pEv>,Advs[e] + (1 - p)E v ,v',Advs[e], and 
2. for each set of actions V" , Ey 1 Advs\ e \ ^ Ey' y" ,Advs[ e ]- 

Proof. The proof of the first item follows the lines of the proof of Proposition 10.5.1; the proof 
of the second item follows from the fact that Qyi C Qyi yn. ■ 
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10.6 Example: Randomized Dining Philosophers 

To illustrate the use of timed progress statements for the analysis of an algorithm, we reconsider 
the randomized dining philosophers algorithm of Lehmann and Rabin, and we show that, under 
the condition that each process has a minimum speed, progress is guaranteed within expected 
constant time. First, we show how to add time to the probabilistic automaton that describes the 
algorithm; then, we add time limitations to the progress statements that we used in Section 6.3.3 
and we derive the upper bound on the expected time for progress; finally we repeat the low 
level proof observing that the coin lemmas are applied in the same way as for the untimed case. 

10.6.1 Representation of the Algorithm 

The probabilistic timed automaton that represent the Algorithm of Lehmann and Rabin can be 
obtained directly from the probabilistic automaton of Section 6.3.2 by adding arbitrary self-loop 
time-passage transition from each state (same as the patient construction of Example 9.2.1). 
Then, in order to enforce a lower bound on the speed of each process, we impose some limitations 
on the adversaries that act on M. For convenience, but without loss of generality, we assume 
that from any point each process in its trying or exit region performs one transition within time 
1. Thus, the adversary schema that we use on M is the set of adversaries A for M such that 
for each finite timed execution fragment a of M, 

1- P-prexeciM^a^frag^iM)} = 1, and 

2. for each element a 1 of £l vrexec (M,A,a) there is no pair of prefixes a\ < a 2 of a'\>a and no 
process i such that process i is in its trying or exit region in Istate(ati) , ltime(a2>oti) > 1, 
and process i does not perform any discrete transition in a.2 >a i- 

We call this adversary schema Unit-Time. 

Remark 10.6.1 Observe that in Condition 1 we require the probability of the admissible 
executions to be 1 rather than requiring the sample space to contain only admissible executions. 
The reason for using probabilities is technical and is due to the fact that the sample space of a 
probabilistic timed executions always contains Zeno timed executions, even though they occur 
with probability 0. From the practical point of view all the Zeno timed executions can be 
ignored. 

In other words, it is not necessary to know the intricacies of the definition of a probabilistic 
timed executions since they are used only to guarantee that the events of interest are measurable. 
From the point of view of verifying the correctness of a randomized distributed algorithm, as 
long as Zeno timed executions occur only with probability 0, it is possible to think that Zeno 
timed executions do not occur at all. ■ 

Remark 10.6.2 (Alternative approach) Another alternative approach to modeling the al- 
gorithm of Lehmann and Rabin, which we do not use here, is to augment the probabilistic 
automaton of Section 6.3.2 with an upper bound for each process i to the time by which pro- 
cess i must perform a transition, and to allow a time-passage transition only when no process 
goes beyond its upper bound. Of course the upper bounds need to be updated opportunely 
within a transition. In this case the condition imposed on an adversary would be just that time 
advances unboundedly with probability 1. ■ 
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10.6.2 The High Level Proof 

The high level proof consists of the same progress statements that we used in Section 6.3.3 
together with a time bound. Specifically, we use the following timed progress statements. 

T --> TIT U C (Proposition 10.6.3), 

-JIT -L, j: u g u -p (Proposition 10.6.15), 
T --> Q U V (Proposition 10.6.14), 

1/2 V ' 

Q -->V (Proposition 10.6.11), 

1/4 V ' 

V -U C (Proposition 10.6.1). 

By combining the statements above by means of Proposition 5.5.3 and Theorem 10.4.1 we 
obtain 

T^C. (10.33) 

1/8 V ; 

Observing that if some process is in the trying region then some process is in the trying region 
unless some process gets to the critical region, we apply Proposition 10.5.1 and we obtain that 
the expected time to reach C from 1ZT is at most 104, i.e., the algorithm of Lehmann and Rabin 
guarantees progress within expected constant time. 

10.6.3 The Low Level Proof 

We now prove the timed progress statements of Section 10.6.2. The proofs are exactly the same 
as the proofs given in Section 6.3.4 with the difference that in this case we consider also time 
bounds and we consider only admissible timed execution fragments since we know that they 
occur with probability 1. 

Proposition 10.6.1 If some process is in P, then some process enters C within time 1, i.e., 

l 

Proof. Let i be the process in P. Then, from the definition of Unit-Time, process i is scheduled 
within time 1, and enters C. ■ 

Lemma 10.6.2 If some process is in its Exit region, then it will enter R within time 3. 

Proof. The process needs to perform two transitions to relinquish its two resources, and then 
one transition to send a rem message to the user. Every adversary of Unit-Time guarantees 
that those three transitions are performed within time 3. ■ 

Proposition 10.6.3 T --^UTuC. 
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Proof. From Lemma 6.3.2, every process that begins in Ep or Es relinquishes its resources 
within time 2 . If no process begins in C or enters C in the meantime, then the state reached 
at this point is a state of 1ZT; otherwise, the starting state or the state reached when the first 
process enters C is a state of C. ■ 

5 . 

We now turn to the proof of Q — ► V . The following lemmas form a detailed cases analysis 

1/4 

of the different situations that can arise in states of Q . Informally, each lemma shows that a 
specific coin event is a sub-event of the properties of reaching some other state. Here we do not 
repeat the proof of Lemma 6.3.4 since it does not depend on timing issues. 

Lemma 10.6.4 

1. Let X 8 _i G {E R , R, F} and X % = W. If FIRST (f lip;_i, left), then, within time 1, 
either X 8 _i = P or Xi = S . 

2. Let Xj-_i = D and Xi = W. If FIRST(f lip 8 _ 1 , left), then, within time 2, either 
X l _ 1 = PorX l = S. *~ 

3. Let Xi_i = S and Xi = W . If FIRST (f lip 8 _ 1 , left), then, within time 3, either X 8 _i = 
P or X t = S. *~ 

4- Let Xi_i = W and Xi = W. If FIRST(f lip 8 _ 1 , left), then, within time 4, either 
X l _ 1 = PorX l = S. *~ 

Proof. The four proofs start in the same way. Let s be a state of M satisfying the respective 
properties of items 1 or 2 or 3 or 4- Let A be an adversary of Unit-Time, and let a be an 
admissible timed execution of £l vrexec (MAs},A) where the result of the first coin flip of process 
i — 1, if it occurs, is left. 

1. By hypothesis and Lemma 6.3.4, i — 1 does not hold any resource at the beginning of a 
and has to obtain Res 8 _2 (its left resource) before pursuing Res 8 _i. From the definition 
of Unit-Time, i performs a transition within time 1 in a. If i — 1 does not hold Res 8 _i 
when i performs this transition, then i progresses into configuration S . If not, it must be 
the case that i — 1 succeeded in getting it in the meanwhile. But, in this case, since i — 1 
flips left, ReSj-_i was the second resource needed by i — 1 and i — 1 therefore entered P. 

2. If Xi = S within time 1, then we are done. Otherwise, process i — 1 performs a transition 
within time 1. Let a = a\ ~ a.^ such that the last transition of a.\ is the first transition 
taken by process i — 1. Then Xi_i(fstate(a2)) = F and Xi(fstate(a2)) = W. Since process 
i — 1 did not flip any coin during a\, from the finite-history-insensitivity of Unit-Time 
and Item 1 we conclude. 

3. If Xi = S within time 1, then we are done. Otherwise, process i — 1 performs a transition 
within time 1. Let a = a\ ~ a.^ such that the last transition of a.\ is the first transition 
taken by process i — 1. If Xi_i(fstate(a2)) = P then we are also done. Otherwise it must 
be the case that Xi_i(fstate(a2)) = D and Xi(fstate(a2)) = W. Since process i — 1 did 
not flip any coin during a\, from the finite-history-insensitivity of Unit-Time and Item 2 
we conclude. 

234 



4. If Xi = S within time 1, then we are done. Otherwise, process i checks its left resource 
within time 1 and fails, process i — 1 gets its right resource before, and hence reaches at 
least state S. Let a = a\ ~ a.^ where the last transition of a.\ is the first transition of a 
that leads process i — 1 to state S. Then Xi_i(fstate(a2)) = S and Xi(fstate(a2)) = W. 
Since process i — 1 did not flip any coin during a\ , from the finite-history-insensitivity of 
Unit-Time and Item 3 we conclude. ■ 

Lemma 10.6.5 Assume that X,_i G {E R ,R,T} and X, = W. If FIRST (f lip;_i, left), 
then, within time 4, either X,_i = P or Xi = S . 

Proof. Follows directly from Lemma 10.6.4 after observing that X,_i G {E R ,R,T} is equiva- 
lent to X 8 _! G {E R , R, F, W, S, D, P}. m 

The next lemma is a useful tool for the proofs of Lemmas 10.6.7, 10.6.8, and 10.6.9. It is just 
repeated from Section 6.3.4. 

Lemma 10.6.6 Let X, G {W, S} or X, G {E R ,R,F,D} with FIRST(f lip,-, left). Further- 
more, let X l+1 G {W, S} or X l+1 G {E R ,R,F,D} with FIRST(fliip i+1 , right). Then the 
first of the two processes i or i + 1 testing its second resource enters P after having performed 
this test (if this time ever comes). 

Proof. By Lemma 6.3.4 Res 8 - is free. Moreover, Res 8 - is the second resource needed by both i 
and i + 1. Whichever tests for it first gets it and enters P. ■ 

Lemma 10.6.7 If Xi = S and Xi + \ G {W, S} then, within time 1, one of the two processes 
i or i + 1 enters P. The same result holds if Xi G {W, S } and X 8 _|_i = S . 

Proof. Being in state S, process i tests its second resource within time 1. An application of 
Lemma 10.6.6 finishes the proof. ■ 

Lemma 10.6.8 Let Xi = S and X l+1 G {E R ,R,F,D}. If FIRST(fliip i+1 , right), then, 
within time 1, one of the two processes i or i + 1 enters P. The same result holds if Xi G 
{E R ,R,F,D}, X i+1 = S^ and FIRST(f lip,-, left). 

Proof. Being in state S, process i tests its second resource within time 1. An application of 
Lemma 10.6.6 finishes the proof. ■ 

Lemma 10.6.9 Assume that X,_i G {E R ,R,T}, X, = W, and X l+1 G {E R ,R,F,W, D}. 
If FIRST(f lip,-_ l7 left) and FIRST(f lip i+1 , right), then, within time 5, one of the three 
processes i — 1, i or i + 1 enters P. 

Proof. Let s be a state of M such that X,_i(s) G {E R ,R,T}, A,(s) = W, and X,_|_i(s) G 
{Er,R,F,W,D}. Let A be an adversary of Unit-Time, and let a be an admissible timed 
execution of Q prexec (M,{s},A) where the result of the first coin flip of process i — 1 is left and 
the result of the first coin flip of process i + 1 is right. By Lemma 10.6.5, within time 4 either 
process i — 1 reaches configuration P in a or process i reaches configuration S in a. If i — 1 
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reaches configuration P, then we are done. If not, then let a = a\ ~ a 2 such that lstate(a\) is 
the first state s' of a with Xi(s') = S . If i + 1 enters P before the end of a\, then we are done. 
Otherwise, Xi + i(fstate(a2)) is either in {W, S } or it is in {E R , R,F, D} and process i + 1 has 
not flipped any coin yet in a. From the finite-history-insensitivity of Unit- Time we can then 
apply Lemma 10.6.6: within time 1 process i tests its second resource and by Lemma 10.6.6 
process i enters P if process i-\-l did not check its second resource in the meantime. If process 
i + 1 checks its second resource before process i does the same, then by Lemma 10.6.6 process 
i + 1 enters P. ■ 

Lemma 10.6.10 Assume that X l+2 G {E R ,R,T}, X l+1 = W, and X % G {E R ,R,F,W, D}. 
If FIRST(f lipj, left) and F/fl5 , T(flip i+2 , right), then, within time 5, one of the three pro- 
cesses i, i + 1 or i + 2, enters P. 

Proof. The proof is analogous to the one of Lemma 10.6.9. This lemma is the symmetric case 
of Lemma 10.6.9. ■ 

Proposition 10.6.11 Starting from a global configuration in Q, then, with probability at least 
1/4, some process enters P within time 5. Equivalently: 

G --+ V. 

1/4 

Proof. Lemmas 10.6.7 and 10.6.8 jointly treat the case where X{ = S and J I+ i G {E R , R, F, #} 
and the symmetric case where X{ G {Er,R,F, #} and X 8 _|_i = S ; Lemmas 10.6.9 and 10.6.10 
jointly treat the case where X{ = W and X 8 _|_i G {Er, R, F,W, D} and the symmetric case 
where X % G {E R , R, F, W, D} and X i+1 = W. 

Specifically, each lemma shows that a compound event of the kind FIRST(f lip,, x) and 
FIRST(f lip •, y) leads to V . Each of the basic events FIRST^fliip}, x) has probability at least 
1/2. From Lemma 6.2.4 each of the compound events has probability at least 1/4. Thus the 
probability of reaching V within time 5 is at least 1/4. ■ 

2 

We now turn to T — ► Q U V . The proof is divided in two parts and constitute the global 

1/2 

argument of the proof of progress, i.e., the argument that focuses on the whole system rather 
than on a couple of processes. 

Lemma 10.6.12 Start with a state s of T . If there exists a process i for which X 8 (s) = F and 
(X 8 _i,X 8 _|_i) 7^ (#,#), then, with probability at least 1/2 a state of Q U V is reached within 
time 1. 

Proof. If s G Q U V, then the result is trivial. Let s be a state of T — (Q U V) and let i be such 
that Xi(s) = F and (X 8 _i,X 8 _|_i) ^ (#,#). Assume without loss of generality that X 8 _|_i ^ #, 

i.e., X 8 _|_i G {E R ,R,F,i^}. The case for X 8 _i ^ # is similar. Furthermore, we can assume 
that X 8 _|_i G {E R ,R,F, D} since if X 8 _|_i G {W, S} then s is already in Q. We show that the 
event schema FIRST((f lip^ left), (flip 8+1 , right)), which by Lemma 6.2.2 has probability 
at least 1/2, leads eventually to a state of Q U V . Let A be an adversary of Unit-Time, and 
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let a be an admissible timed execution of £l vrexec (MAs},A) where if process i flips before process 
i + 1 then process i flips left, and if process i + 1 flips before process i then process i + f flips 
right . 

Then, within time f, i performs one transition and reaches W. Let j G {i,i + 1} be the 
first of i and i + f that reaches IV and let si be the state reached after the first time process j 
reaches W. If some process reached P in the meantime, then we are done. Otherwise there are 
two cases to consider. If j = i, then, flip, gives left and X 8 (si) = W whereas X 8 _|_i is (still) 
in {Pr, R, F, D}. Therefore, si G Q . If j = i + 1, then f lip 8+1 gives right and X 8 _|_i(si) = W 
whereas X 8 (si) is (still) F. Therefore, si G Q. ■ 

Lemma 10.6.13 Start with a state s of J 7 . If there exists a process i for which X 8 (s) = F and 
(X 8 _i(s),X 8 _|_i(s)) = (#,#). Then, with probability at least 1/2, a state of Q U V is reached 
within time 2. 

Proof. The hypothesis can be summarized into the form (X 8 _i(s),X 8 (s),X 8 _|_i(s)) = (#,P, #). 
Since i — 1 and i-\-l point in different directions, by moving to the right of i + 1 there is a process 
k pointing to the left such that process k + 1 either points to the right or is in {Er,R,F,P}, 
i.e., X k (s) G {W, £, D] and X k+1 (s) G {^, iE, F, W, 5 , ^ , P}. 

If Xjt(s) G {W, 5*} and Xfc+i(s) 7^ P then s £ Q and we are done; if Xk+i(s) = P then 
s G P and we are done. Thus, we can restrict our attention to the case where Xk(s) = D. 

We show that P/P5T((flip fc , left), (flip fc+1 , right)), which by Lemma 6.2.2 has proba- 
bility at least 1/2, leads to QUV within time 2. Let A be an adversary of Unit-Time, and let a 
be an admissible timed execution of £l vrexec (MAs},A) where if process k flips before process k-\-l 
then process k flips left, and if process k-\-l flips before process k then process k-\-l flips right. 

Within time 2 process k performs at least two transitions and hence goes to configuration 
W. Let j G {k,k + 1} be the first of k and k + 1 that reaches W and let si be the state 
reached after the first time process j reaches W. If some process reached P in the meantime, 
then we are done. Otherwise, we distinguish two cases. If j = k, then, f lipj. gives left and 
Xk(si) = W whereas X k+ i is (still) in {Er,R,F, #}. Thus, si G Q. If j = k -\- 1, then flip fc+1 
gives right and Xfc+i(si) = W whereas Xk{s\) is (still) in {D,F}. Thus, si G Q. ■ 

Proposition 10.6.14 Start with a state s of T . Then, with probability at least 1/2, a state of 
Q U V is reached within time 2. Equivalently: 

T -^ Q U P. 
1/2 

Proof. The hypothesis of Lemmas 10.6.12 and 10.6.13 form a partition of T . ■ 

Finally, we prove PT — ► T U Q U P. 

Proposition 10.6.15 Starting from a state s of 1ZT, then a state of T U Q U P is reached 
within time 3 Equivalently: 

UT -^ PU£ UP. 
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Proof. Let s be a state of 1ZT. If s G T U Q U V, then we are trivially done. Suppose that 
s $l T U Q U P. Then in s each process is in {Er, R, W, S, D} and there exists at least process 
in {W, S, D}. Let A be an adversary of Unit-Time, and let a be an admissible timed execution 

°f ^prexec(M,{s},A)- 

We first argue that within time 1 some process reaches a state of {S,D,F} in a. This 
is trivially true if in state s there is some process in {S,D}. If this is not the case, then all 
processes are either in Er or R or W . Eventually, some process in R or W performs a transition. 
If the first process not in Er performing a transition started in Er or R, then it reaches F and 
we are done; if the first process performing a transition is in W, then it reaches S since in s no 
resource is held. Once a process i is in {S,D,F}, then within time 2 process i reaches either 
state F or P, and we are done. ■ 

10.7 Abstract Complexity Measures 

We have seen how to measure the expected time to satisfy a property. However, the technique 
can be extended to other kinds of measures of complexity. Specifically, let (f> be a complexity 
measure on timed execution fragments that is additive under concatenation, i.e., (f>(q\ ~ #2) = 
4>{q\) + (f>(q2)- Then we can compute the expected (f> rather than the expected time, where the 
<f> of a state q of H is defined to be cf)(q>qQ). We generalize the notation for timed progress 
statements by writing 

U ^Aivs U' (10.34) 

with the meaning that Fr Advs ,u( e u' ,cf>(c)) ^ Vi where the event schema eu> ^i c \ applied to a timed 
probabilistic execution fragment H returns the set of timed executions a of Q,jj where a state 
from U' is reached within complexity c. More specifically, let Conesui^i c \(H) be the set of 
minimal timed execution fragments q of M such that C^ is not empty, Istate(q) £ U', and 
(f>(q>qo) < c Then, e ul}4> ^(H) = ^qeGo n e Su , A(c) (H)Cq. Observe that time is just one of the 
possible complexity measures. 

The same definition can be extended to sets of actions as we have done previously, and the 
concatenation theorem is still valid. 

The expected complexity of a finitely satisfiable event schema can be defined easily. Specifi- 
cally, if e is a finitely satisfiable event-schema and Cones(H) identifies the points of satisfaction 
of e, then for each probabilistic timed execution fragment H of M we define EH,<f>[e], the ex- 
pected complexity to satisfy e in if , as follows. 

E fe | = (E !£ Mfl) P ff[ C JWrf)) ifP H [e(H)] = l 
1 00 otherwise. 

Then, a proposition similar to Proposition 10.5.1 can be proved. 

Proposition 10.7.1 Suppose that 

{ TltH TT> 

J U ^Advs v (10.36) 

I U => (U Unless U'), 
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Figure 10-1: An example of the use of £. 

and suppose that Advs is finite history insensitive and that sS ^ &Ms) f or eac h A £ Advs and 
each s £ U. Then, 

Eu,Advs,<t>[ e ] ^ C + P E U',AdvsA e ] + ( 1 ~ P)(i + E U,Adv S ,<f>[ e ])' (10.37) 

where 

£ = SUP q et-frag*(M)\l S tate(q)eU ( SU Pq'>q (*«/g»|g<g»< 3 '(0(?">?)))) • (10.38) 

Proof. This proof has the same structure as the proof of Proposition 10.5.1. Here we describe 
in detail only the main differences. In particular, we show part of the derivation from Equa- 
tion (10.16) to Equation (10.21), where the constant £ is used. Observe that if we use (f> to 
express time complexity, then £ = 0. 

From (10.35) the expected complexity for success for e is 

Eh,M= E PH[C q ]<t>{q><^). (10.39) 

gG Cones(H) 

For each d > 0, let Cones d be a function that expresses the event of reaching complexity d as 
a union of disjoint cones. From the definition of a probabilistic timed execution, we know that 
Cones d exists and, from (10.38), we know that for each probabilistic timed execution fragment 
H and each q £ Conesd(H), d < <j>(q>qo) < d + £. Let e be any positive number. Following 
the same derivation as in the proof of Proposition 10.5.1, we obtain 

E H ,M < ( c + e ) f E p H[C q ]E W)AdvS)4> [e])\ + f J2 Ph[CM + e uMv'M)\ -(10-40) 
\9e01 / \qee 2 J 

■ 
One of the novel aspects of Proposition 10.7.1 is the constant £. Roughly speaking, £ gives us a 
lower bound to the minimum complexity increase that we can obtain by moving along a timed 
execution fragment. 

Example 10.7.1 (Why £ is necessary) For example, if the abstract complexity that we use 
is the number of discrete actions that appear in a timed execution fragment, then £ = 1. In fact, 
whenever we perform a discrete action, the complexity increases by 1. Figure 10-1 shows an 
example where £ = 1 and where Equation (10.37) is invalidated if we do not include £. Denote 
the probabilistic timed execution fragment of Figure 10-1 by H . Let U be {so} 5 U' be {si}, and 
let e express the property of reaching U'. Let Advs contain only one adversary that generates H 
when applied to so- Let (f> count the number of external actions in a timed execution fragment 

(no time-passage actions in H). Then, it is immediate to verify that the statement U — >U' is 
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valid in H and that also U =>■ (U Unless U') is valid. By applying Equation (10.37) with £ = 1, 
we obtain 

Eu,Advs,M < t + V2(l + E UAdvSt4> [e]), (10.41) 

which leads to Eu,Advs,<f>[ e ] < 3. If we did not use £ in Equation (10.37) we would have obtained 
Eu,Advs,4>[ e ] < 2. We now show that Efj^e] = 3. In fact, 

Eh,M = 1 + ^\ + ^ + ^ + --- (10 ' 42) 

By rearranging the terms, we obtain 

E H Je} = Y -(- + - + - + — + ■■). (10.43) 

' 0L J ^2 8 V2 4 8 16 J v ; 

Recall that J2i>o 1/2 8 = 2. Thus, by rearranging the terms again, 

E Hi4> [e] = 2 + 1/2 Q + i + i + ^ + • • •) = 3. (10.44) 

Roughly speaking, the transition relation of H is structured in such a way that whenever the 
experiment of reaching U' from U fails, the system looses one additional complexity unit during 
the random draw. In the proof of Proposition 10.7.1 this phenomenon is detected when we define 
the partition ©i and ©2- To make sure that ©i and ©2 partition an event with probability 1 
and that ©i captures all the places where U' is reached within time t, ©2 must be based on 
states reached after time t. In the probabilistic execution H of this example the states of ©2 
have complexity t + 1. ■ 

10.8 Example: Randomized Agreement with Time 

Using abstract complexity measures it is possible to show that the randomized agreement 
algorithm of Ben-Or guarantees agreement within an expected exponential time. This is not 
an exceptional complexity result, but it corresponds to the time complexity of the algorithm. 

In more detail, we add time to the probabilistic automaton that describes Ben-Or's protocol 
in the same way as we have done for the Dining Philosophers algorithm of Lehmann and Rabin. 
In this case each adversary is required to schedule every process that enables some transition 
within time 1 from every point. Then we show an upper bound linear in st on the time it 
takes to all processes to complete a specific stage st. Finally, we derive an upper bound on 
the expected number of stages it takes for all processes to decide. This is achieved by defining 
an abstract complexity on the timed executions of M that checks the highest stage reached at 
every point. A direct extension of the untimed proof without abstract complexities would not be 
possible. In fact, given a reachable state s, the validity of the progress statement of Chapter 6 
relies on completing the highest stage reached in s, and we cannot establish any useful upper 
bound on the time to complete such stage: there is no useful bound on the difference between 
the highest and the lowest stages reached in s, and the adversary may stop the processes with 
the highest values of st. We start by proving the upper bound on the time it takes to each 
process to complete some stage st. 
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Lemma 10.8.1 There is a constant d such that, for each stage st, each process completes stage 
st within time d ■ st. 

Proof. Let d\ be the maximum time it takes to each process from the moment it reaches a new 
stage st to the moment it broadcasts its value and its value is delivered; let di be the maximum 
time it takes to each process to broadcast and deliver its second message after receiving enough 
messages from the first round; let d^ be the maximum time it takes to each process to move to a 
new stage once it has received enough messages from the second round. Then d = d\ + di + d^. 
Since we have not defined formally M, we cannot say explicitly what is the value of d. 

We show the result by induction on st where for the base case we assume that st = 
and that stage is completed by time 0. By induction, by time d ■ st each non-faulty process 
has completed round st. Then, by time d\ + d ■ st each non-faulty process has broadcasted 
and delivered its first round message, and thus every non-faulty process has received enough 
messages for the first round of stage st + 1. Within additional time di each non-faulty process 
delivers its second message, and within additional time d^ each non-faulty process reaches stage 
st + 2, i.e., within time d(st + 1) each non-faulty process completes stage st + 1. ■ 

For each finite timed execution fragment a of M define (f>(ot), the stage complexity of a, to 
be max-stage(lstate(a)) — max-stage(fstate(a)) , where for each state s, max-stage(s) is the 
maximum stage that is reached in s by some process. Observe that this complexity measure is 
an upper bound to the stage at which some process decides since if at state s the first process 
has just decided, then max-stage(s) is not smaller than the stage of the process that has decided. 
Thus, an upper bound on the expected (f> for the decision of the first process is an upper bound 
on the expected stage at which the first process decides. We show the following two statements. 

B%, r fU0. (10.45) 

T*Mo. (10.46) 

1/2" 

Then, by combining (10.45) and (10.46) with Theorem 5.5.2, we obtain 

B^O. (10.47) 

1/2" 

From Proposition 10.7.1, we obtain 

EB,Unit-Time,4>[eo] < 3 + (1 - 1/2™)(1 + Eg, Unit-Time,<t>[ e o\), (10.48) 

where 1 is the value of £ given by (10.38). By solving Equation (10.48) we obtain 

E B ,Umt-n m eA e o] < 2 n+2 - 1. (10.49) 

Since if a process decides at stage st then each other non-faulty process decides within stage 
st + 1, then we can derive that the expected stage by which every process decides is at most 
2 n+2 , and thus, from Lemma 10.8.1, each process decides within expected time d ■ 2 n+1 . 

The proofs for (10.45) and (10.46) have the same structure as the corresponding proofs 
for the untimed case. Recall that the proof of (10.45) consider the maximum stage st of a 
reachable state s and states that eventually stage st + 1 is reached, at which time a state of T 
is reached. The proof of (10.46) states that a specific coin lemma leads a process to decide by 
stage max-stage(s) + 1. Then, since if a process decides a stage st each process decides by stage 
st + 1, the complexity of the state where the first process decides is at most max-stage(s) + 2. 
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10.9 Discussion 

To our knowledge this is the first time that statements similar to our timed progress statements 
have been used for the analysis of the performance of a randomized distributed algorithm. In 
particular, we have been able to prove similar results only because we have studied techniques to 
prove properties that hold with some probability different than 1. This should be a sufficiently 
strong reason to pursue additional research on methodologies (automatic or not) for the analysis 
of properties that hold with probabilities different than 1. The work of Hansson [Han94] and 
the algorithm that Courcoubetis and Yannakakis present in [CY90] are in this direction. 
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Chapter 11 

Hierarchical Verication Timed 
Trace Distributions 

11.1 Introduction 

In this chapter we extend the trace distribution preorder of Chapter 7 to the timed framework. 
The main difference is that we use timed traces rather than traces. A timed trace contains the 
sequence of discrete actions that occur within a timed execution plus the time of occurrence 
of each action and the time at which the observation ends. That is, in a timed execution we 
observe at what time each external action occurs and, if finitely many actions occur, how much 
time elapses after the occurrence of the last action. 

We define a preorder relation based on timed trace distribution inclusion, and we characterize 
the coarsest precongruence that is contained in the timed trace distribution preorder by using 
a timed principal context, which is just the principal context of Chapter 7 augmented with 
arbitrary time-passage self-loop transitions from its unique state. Most of the proofs follow 
directly from the results already proved in Chapter 7, since in several cases it is sufficient to 
study ordinary trace distributions in order to derive properties of timed trace distributions. 

11.2 Timed Traces 

We start by defining the main object of observation, i.e., timed traces. The definition of a timed 
trace that we give in this section is taken directly from [LV95]. 

Timed Sequence Pairs 

Let K be any set that does not intersect !R + . Then a timed sequence over K is defined to be a 
(finite or infinite) sequence 7 over K X J?-° in which the time components are nondecreasing, 
i.e., if (k,t) and (k',t') are consecutive elements in 7 then t < t' . We say that 7 is Zeno if it is 
infinite and the limit of the time components is finite. 

A timed sequence pair over K is a pair (3 = (7,2), where 7 is a timed sequence over K and 
t G 3£-° U {00}, such that t is greater than or equal to all time components in 7. We write 
seq(P), and Itime(ft) for the two respective components of (3. We denote by tsp(K) the set of 
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timed sequence pairs over K . We say that a timed sequence pair (3 is finite if both seq{(3) and 
Itime(p) are finite, and admissible if seq{(3) is not Zeno and Itime(ft) = oo. 

Let (3 and /3' be timed sequence pairs over K with (3 finite. Then define (3; (3' to be the timed 
sequence pair (seq(P)j, ltime{(3) + Itime(P')), where 7 is the modification of seq{(3') obtained 
by adding ltime{(3) to all the time components. If (3 and (3' are timed sequence pairs over a set 
K , then (3 is a prefix of /3', denoted by /3 < /3', if either /3 = (3' , or /3 is finite and there exists a 
timed sequence pair (3" such that (3' = (3; (3". 

Lemma 11.2.1 < is a partial ordering on the set of timed sequence pairs over K . ■ 

Now we describe how to translate from a sequence over K U !K + , and ordinary trace, to a timed 
sequence pair over K. First, if (3 is any sequence over K U !R + , then we define the time of 
occurrence of any ii'-element in (3 to be the sum of all the reals that precede that element in 
(3. We also define ltime{(3) to be the sum of all the reals in (3. Finally, we define t-trace(fi) to 
be the timed sequence pair (7, Itime(P)), where 7 is the subsequence of (3 consisting of all the 
elements of K , each paired with its time of occurrence. 

If (3 is a sequence over K U ?R. + then we say that (3 is admissible if the sum of the positive 
reals in (3 is infinite. 

Lemma 11.2.2 If (3 is a finite or admissible timed sequence pair then t-trace(trace{(3)) = (3. ■ 

Lemma 11.2.3 If (3 is a sequence over K U ?R. + then (3 is admissible if and only if t-trace(fi) 
is admissible. ■ 

Timed Traces of Timed Probabilistic Automata 

Suppose that a = cJoai^ia2^2 • • • is a timed execution fragment of a timed probabilistic au- 
tomaton M. For each a,-, define the time of occurrence t{ to be J2j<i Itime(ujj), i.e., the sum of 
the lengths of all the trajectory intervals preceding a 8 - in a. Let 7 be the sequence consisting of 
the actions in a paired with their times of occurrence: 

7 = (a 1 ,t 1 )(a 2 ,t2)---- 
Then t-trace(a), the timed trace of a, is defined to be the pair 

(7 \ (vis(M) X K+), Itime(aj). 

Thus, t-trace(a) records the occurrences of visible actions together with their times of oc- 
currence, and together with the time spanned by a. Note that neither internal actions nor 
time-passage actions appear explicitly in the timed trace of a. 

Proposition 11.2.4 If a is a timed execution fragment of M then t-trace(a) is a timed se- 
quence pair over vis(M). ■ 

Proposition 11.2.5 Let a be a timed execution fragment of M , and let trace(a) denote the 
ordered sequence of external actions that appear in a. Then, t-trace(a) = t-trace(trace(a)). ■ 

Proposition 11.2.6 If a = a\ ~ a 2 is a timed execution fragment of M , then t-trace(a) = 
t-trace(ai); t-trace(a2) ■ ■ 
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We write t-traces(M) for the set of all timed traces of M, t-traces*(M) for the set of finite 
timed traces of M, and t-traces°°(M) for the set of admissible timed traces of M, 

The timed traces of a probabilistic timed automaton M can be characterized also in terms 
of its time-enriched executions or in terms of its ordinary executions. Specifically, if a is a time- 
enriched execution of M, then let t-trace(a) denote t-trace(t-exec(a)) , and if a is an execution 
of M, then let t-trace(a) denote t-trace(trace(a)) . The following proposition holds. 

Proposition 11.2.7 Let M be a probabilistic timed automaton. 

l.Ifa is a time-enriched execution of M , then there is a timed execution a' of M such that 
t-trace(a) = t-trace(a'). 

2. If a is a timed execution of M , then there is a time-enriched execution a' of M such that 
t-trace(a) = t-trace(a'). 

3. If a is a timed execution of M , then there is an execution a' of M such that t-trace(a) = 
t-trace(a'). 

4- If a is an execution of M , then there is a timed execution a' of M such that t-trace(a) = 
t-trace(a'). 

Proof. 

1. Let a' be t-exec(a). Then, t-trace(a) = t-trace(a') by definition. 

2. Let a be uJoaiUJia2 ■ ■ ■■ If a is a finite timed execution or an infinite sequence, then let 

a' = fstate(ujo) ~ a\ ~ a.^ ~ • • •, where for each i, 



a; 



tjji-iaifstate(oji) if cj 8 _i has domain [0,0], 

fstate(oJi_i)ltime(oJi_i)tjJi_iaifstate(oJi) otherwise; 



if a = uJoaiUJia2 ■ ■ -a n ijj n and the domain of uj n is right-open, then let a' = fstate(ujo) ~ 
a.\ ~ • • • ~ a n ~ ot' n+1 , where the a 8 's are defined above and a' n+1 = a^dio^c^^ ' ' ' 1S an 
infinite sequence such that uj'qUj'-^uj^ • • • = u n . It is immediate to verify that a and a' have 
the same timed trace since a = t-exec(a'). 

3. Let a be uJoaiUJia2 ■ ■ ■■ If a is a finite timed execution or an infinite sequence, then let 

a' = fstate(ujo) ~ a\ ~ a.^ ~ • • •, where for each i, 



at; 



lstate(oJi_i)aifstate(oJi) if cj 8 _i has domain [0,0], 

fstate(oJi_i)ltime(oJi_i)lstate(oJi_i)aifstate(oJi) otherwise; 



if a = uJoaiUJia2 ■ ■ ■ a n uj n and the domain of uj n is right-open, then let a" = fstate(ujo)"" a\^ 
■ ■ -^ a n ~ a' n+1 , where the a 8 's are defined above and a' n+1 = fstate(oj n )ditjj n (di)d2tjJ n (di + 
(I2) ■ ■ ■ is an infinite sequence such that J2i di = ltime(uj n ). It is immediate to verify that 
a and a' have the same timed trace. 
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4. Given a = soaiSi«2 • • •, build a time-enriched execution a" by replacing each state s 8 - with 
a trajectory for (s 8 _i, a 8 , s 8 ) whenever a 8 - is a time-passage action. Then, t-trace(a) = 
t-trace(a"). Item 2 is enough to conclude. ■ 

The bottom line of the proposition above is that for the study of the timed traces of a probabilis- 
tic timed automaton it is not necessary to observe the trajectories spanned by a computation. 
The points of occurrence of discrete actions are sufficient. 

11.3 Timed Trace Distributions 

In this section we define the timed trace distributions of a probabilistic timed automaton and we 
extend the action restriction operation. The main result is that it is possible to study the timed 
trace distributions of a probabilistic timed automaton M by considering either its probabilistic 
executions, or its probabilistic time-enriched executions, or its probabilistic timed executions. 

11.3.1 Three ways to Define Timed Trace Distributions 

We now define the timed trace distribution of a probabilistic execution, of a probabilistic time- 
enriched execution, and of a probabilistic timed execution of a probabilistic timed automaton. 
The definitions are given in the same style as for the untimed case. Furthermore, we show that 
the three definitions lead to the same collection of timed trace distributions. This enforces the 
remark that for the study of the timed trace distributions of a probabilistic timed automaton 
it is not necessary to observe the trajectories spanned by a computation. 

Timed Trace Distribution of a Probabilistic Execution 

Let if be a probabilistic execution of a probabilistic timed automaton M, and let / be a function 
from CIh to = tsp(vis(M)) that assigns to each extended execution its timed trace. The timed 
trace distribution of H , denoted by t-tdistr(H), is the probability space completion((Q,,J-,P)) 
where T is the u-field generated by the cones Cp, where (3 is a finite timed sequence pair of 
tsp(vis(M)), and P = /(Pff). Note that from Proposition 3.1.4 / is a measurable function 
from (n H , Th) to(fi,.F). 

Timed Trace Distribution of a Probabilistic Time-Enriched Execution 

Let if be a probabilistic time-enriched execution of a probabilistic timed automaton M, and 
let / be a function from 0# to = tsp(vis(M)) that assigns to each time-enriched extended 
execution its timed trace. The timed trace distribution of if, denoted by t-tdistr(H), is the 
probability space (0, J 7 , P) where T is the u-field generated by the cones Cp, where (3 is a finite 
timed timed sequence pair of tsp(vis(M)), and P = /(Pff). Note that from Proposition 3.1.4 
/ is a measurable function from {£Ih-,J~h) to (0, J 7 ). 

Timed Trace Distribution of a Probabilistic Timed Execution 

Let if be a probabilistic timed execution of a probabilistic timed automaton M, and let / 
be a function from 0# to = tsp(vis(M)) that assigns to each timed extended execution 
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its timed trace. The timed trace distribution of H , denoted by t-tdistr(H), is the probability 
space (S7,.F, P) where T is the u-field generated by the cones Cp, where (3 is a finite timed 
timed sequence pair of tsp(vis(M)), and P = /(Pff). Note that from Proposition 3.1.4 / is a 
measurable function from (£Ih,3~h) to (0, J 7 ). 

Equivalence of the Definitions 

We now show that the three definitions of a timed trace distribution lead to the same collection 
of timed trace distributions when applied to a probabilistic timed automaton (cf. Proposi- 
tions 11.3.2 and 11.3.4). Thus, we can freely denote a generic timed trace distribution by V 
and denote the timed trace distributions of a probabilistic tomed automaton M by t-tdistrs(M). 

Lemma 11.3.1 Let H be a probabilistic time- enriched execution of a probabilistic timed au- 
tomaton M . Then, t-tdistr(H) = t-tdistr(sample(H)) . 

Proof. Let V be t-tdistr(H) and let V be t-tdistr(sample(H)) Consider a finite timed trace 
(3. From the definition of t-tdistr(), 

Pv<[Cp] = P sa mple(H)[{a G ^sample(H) I P < t-trace(a)}] . (11.1) 

Since Cp is a finitely satisfiable event, there is a set of of states of sample(H) such that for 
each element q of 0, (3 < t-trace(q), and such that 

{« e £l sample(H) | fi < t-trace(a)} = U qe@ C* am ^ H \ (11.2) 

Thus, 

Pv>[Cp] = £ P sampH H)[C s q amp ' e(H) ]- ( n - 3 ) 

gee 

From Equation (9.55), Equation (11.3) becomes 

Pv\C p ]= J2 Ph[C?]. (11.4) 

q£sample~ (G) 

Observe that sample -1 (0) is a characterization of Op for P, and thus, 

Pv[Cp] = Pv[Cp\. (11.5) 

This completes the proof. ■ 

Proposition 11.3.2 Let M be a probabilistic timed automaton. Then, for each probabilis- 
tic time-enriched execution H of M there exists a probabilistic execution H' of M such that 
t-tdistr(H) = t-tdistr(H'), and for each probabilistic execution H of M there exists a proba- 
bilistic time-enriched execution H' of M such that t-tdistr(H) = t-tdistr(H'). 

Proof. Follows directly from Propositions 9.3.6 and 9.3.7, and from Lemma 11.3.1. ■ 

Lemma 11.3.3 Let H be a probabilistic time-enriched execution of a probabilistic timed au- 
tomaton M . Then, t-tdistr(H) = t-tdistr(t-sample(H)) . 
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Proof. Let V be t-tdistr(H), and let V be t-tdistr(t-sample(H)) . Consider a finite timed 
sequence pair V of tsp(vis(M)). From the definition of t-tdistr, 

P v [Cp] = P H [{a £ &h I P < t-trace(a)}]. (11.6) 

From the definition of t-exec{Vjj)i 

Pv[Cp] = Pt-exec(V H )[{a e ^t-exec(H) I P < t-trace (a)}] . (11.7) 

With a similar analysis, 

Pv<[Cp] = Pt-sample(H)[{a G ^t-sample(H) I P < t-trace(a)}] . (11.8) 

Since from Proposition 9.3.11 t-exec{Vjj) = 7~ > t-sam]>le(H)i an( i si nce the events of (11.7) and (11.8) 
are unions of countably many disjoint cones, we conclude that Pv[Cfj] = Pv[Cfj]- ■ 

Proposition 11.3.4 Let M be a probabilistic timed automaton. Then, for each probabilistic 
time-enriched execution H of M there exists a probabilistic timed execution H' of M such that 
t-tdistr(H) = t-tdistr(H'), and for each probabilistic timed execution H of M there exists a 
probabilistic time-enriched execution H' of M such that t-tdistr(H) = t-tdistr(H'). 

Proof. Follows directly from Propositions 9.3.8 and 9.3.9, and from Lemma 11.3.3. ■ 

Proposition 11.3.5 Let Hi and if 2 be two equivalent probabilistic time- enriched executions of 
a probabilistic timed automaton M . Then, t-tdistr(H\) = t-tdistr(H2). 

Proof. From Proposition 9.3.10, t-sample(Hi) = t-sample^R^)^ an d from Lemma 11.3.3, 
tdistr(H\) = tdistr(t-sample(Hi)) and tdistr(H2) = tdistr(t-sample(H2)). Thus, combining 
the observations above, t-tdistr(H\) = t-tdistr(H2). ■ 

11.3.2 Timed Trace Distribution of a Trace Distribution 

Given a trace distribution of a probabilistic timed automaton, it is possible to define its timed 
trace distribution as we have done for ordinary traces. Thus, let V be a trace distribution of a 
probabilistic automaton, and let / be a function from O-p to = {t-trace(P) \ P £ £lv} that 
assigns to each trace its timed trace. The timed trace distribution of V, denoted by t-tdistr(V), 
is the probability space completion((Q,,J-,P)) where T is the u-field generated by the cones 
C/3, where /3 is a finite timed trace, and P = /(P-p). Note that from Proposition 3.1.4 / is a 
measurable function from (O-p, ^"-p) to (0, J 7 ). 

Proposition 11.3.6 Let H be a probabilistic execution of a timed probabilistic automaton M. 
Then, t-tdistr(H) = t-tdistr(tdistr(H)). 

Proof. Let V be t-tdistr(H), and let V be t-tdistr(tdistr(H)). We show first that V and V 
have the same sample space. Then, we show that they assign the same probability to each cone. 
To show that V and V have the same sample space, it is enough to show that for each 
timed sequence pair /3 of tsp(vis(M)) thehre is a trace P' of ext(M)* U ext(M) w such that 
t-trace(P') = p. Let (/3 = (a,i,ti)(a,2,t2), (^3,^3) • • •,£)• If seq(P) is an infinite sequence, then 
let P' = P1P2P3 ■ ■ •, where for each i, if i,- + i = ti, then Pi = a,-, and if i,- + i > ti, then Pi = 
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cii(ti + i — ti). If seq(fi) is a finite sequence, i.e., seq{(3) = (ai,ti)(a2,t2), ( a 3 5 ^3) • • •, (a>n,t n ) then 
[3' = /?i/?2/?3 • • ■ (3 n -i(3' n where the /3 8 's are defined above, and (3' n is a n if i n = t, a n (t — t n ) if 
< t — t n < oo, and a n followed by the infinite sequence of l's if t = oo. It is easy to verify 
that in every case t-trace{(3') = (3. 

To show that V and V assign the same probability to each cone, let (3 be a finite timed 
trace. From the definition of t-tdistr and tdistr, 

P v ,[Cp] = P H [{a e Off I (i < t-trace(trace(a))}]. (11.9) 
From Proposition 11.2.5, (11.9) becomes 

P v ,[Cp] = P H [{a £ Off I (3 < t-trace(a)}], (11.10) 

which is the definition of Pv[Cfj]. ■ 

11.3.3 Action Restriction 

Finally, we extend the action restriction operator to timed trace distributions. Let M be a 
probabilistic timed automaton, and let V be a set of visible actions of M. For each timed trace 
[3 = (7,/) of M, let (3 \ V be the pair (7', i) where 7' is obtained from 7 by removing all the 
pairs whose action is in V. Let V be a timed trace distribution of M. Define P f "K to be the 
timed trace distribution (0,.F, P) where = Op [" V, T is the u-field generated by the cones 
C/3, where (3 is a finite timed trace, and P = Pj> [" F. Note that from Proposition 3.1.4 \ V is a 
measurable function from (Ox>,.Fx>) to (0,.F). Action restriction commutes with the operation 
of taking a timed trace distribution of a trace distribution. 

Proposition 11.3.7 Let V be a trace distribution of a probabilistic timed automaton M , and 
let V be a set of visible actions of M . Then, t-tdistr {V \ V) = t-tdistr (V) \ V. 

Proof. Let V be t-tdistr {V \ V), and let V" be t-tdistr {V) \ V . Let (3 be a finite timed trace. 
By applying the definitions of t-tdistr and of \, we obtain the following two equations. 

PwiCp] = Pv[{f3' e Op I (3 < t-trace((3' \ V)}]. (11.11) 

P v „[Cp] = P v [{{3' e Op I (3 < t-trace{fl r ) \ V}}. (11.12) 

Observe that for each (3' of Op, t-trace{(3' \ V) = t-trace(p') \ V. Thus, the right expressions 
of (11.11) and (11.12) denote the same value. That is, Pv'[Cfj] = Pv"[Cfj]- ■ 

11.4 Timed Trace Distribution Precongruence 

Let Mi,M2 be two probabilistic timed automata with the same external actions. The timed 
trace distribution preorder is defined as follows. 

Mi Q Dt M 2 iff t-tdistrsiMi) C t-tdistr s{M 2 ). 

As for the untimed case, the timed trace distribution preorder is not a precongruence. A 
counterexample can be created directly from the counterexample of Chapter 7 by augmenting 
the probabilistic automata of Figure 7-4 with arbitrary self-loop time-passage transitions from 
their deadlock states (the states that do not enable any transition). Thus, we define the 
timed trace distribution precongruence, denoted by QDCt, as t ne coarsest precongruence that is 
contained in the timed trace distribution preorder. 
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11.5 Alternative Characterizations 

The timed trace distribution precongruence can be characterized by a timed version of the 
principal context of Chapter 7. Namely, let the timed principal context, denoted by Cp be 
the principal context of Figure 7-6 augmented with self-loop time-passage transitions for each 
time-passage action d. Then, the following holds. 

Theorem 11.5.1 M x Q DCt M 2 iff Mi\\C P Q Dt M 2 \\C P . 

Thus, if we define the principal timed trace distributions of a probabilistic timed automaton 
M, denoted by pt-tdistrs(M), to be the timed trace distributions of M||Cp, then we get the 
following. 

Corollary 11.5.2 M x Edc* M 2 iff ext(Mi) = ext(M 2 ) and pt-tdistrs(Mi) C pt-tdistrs{M 2 ). 

■ 

The rest of this section is dedicated to the proof of Theorem 11.5.1. The structure of the proof 
follows the same lines as the proof of Theorem 7.5.1, where only one additional transformation 
step is added: a distinguishing context is transformed into a new time-deterministic context 
where each state enables either discrete actions only or time-passage actions only. A time- 
deterministic context is a probabilistic automaton such that for each state s and each time- 
passage action d, if s — ► si and s — ► s 2 , then si = s 2 . All the lemmas except for one are 
proved by reducing the problem to the untimed framework. 

Lemma 11.5.3 Let C be a distinguishing context for two probabilistic timed automata Mi and 
M 2 . Then there exists a distinguishing context C" for Mi and M 2 with no discrete actions in 
common with Mi and M 2 . C" is called a separated context. 

Proof. The context C" is built from C in the same way as in the proof of Lemma 7.5.3. The con- 
structions clp and exch work as well (they never exchange transitions involving time-passage), 
and the proof is carried out at the level of probabilistic executions rather than probabilistic 
timed executions. 

Specifically, let V be a timed trace distribution of Mi||C that is not a timed trace distri- 
bution of M2IIC. Consider a probabilistic execution Hi of Mi||C such that t-tdistr(Hi) = V, 
and consider the scheduler that leads to Hi. Apply to Mi||C" the same scheduler with the 
following modification: whenever a transition ((si,c),a,Vi ® V) is scheduled in Mi||C, sched- 
ule ((si, c), ai, V((si, c'))), where d is C( ca ^), followed by ((si, c'), a, Vi ® V(c')), and, for each 
s[ G Oi, followed by ((s^, c'), a 2 ,V(s / 1 )(y)T > ). Denote the resulting probabilistic execution by H[ 
and the resulting timed trace distribution by V . From Lemma 7.5.3, tdistr(Hi) = tdistr(H[) \ 
vis(M 1 \\C), and thus, from Propositions 11.3.6 and 11.3.7, V = V \ vis{M 1 \\C). 

Suppose by contradiction that it is possible to obtain V from M2IIC". Consider the scheduler 
that leads to V in M2IIC", and let H' 2 be the corresponding probabilistic execution. Then, from 
Lemma 7.5.3, clpiexchiH^)) is a probabilistic execution of M2IIC", and tdistr(clp(exch(H2))) = 
tdistr(H' 2 ) \ acts(Mi\\C). From Propositions 11.3.6 and 11.3.7, V = t-tdistr(clp(exch(H'2))), 
which is a contradiction. ■ 

Lemma 11.5.4 Let C be a distinguishing separated context for two probabilistic timed automata 
Mi and M 2 . Then there exists a distinguishing cycle-free separated context C" for Mi and M 2 . 
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Proof. The context C" can be built by unfolding C . Every scheduler for C can be transformed 
into a scheduler for C" and vice versa, leading to the same timed trace distributions. ■ 

Lemma 11.5.5 Let C be a distinguishing cycle-free, separated context for two probabilistic 
timed automata Mi and Mi- Then there exists a distinguishing time- deterministic, cycle-free 
separated context C" for Mi and Mi that from any state enables either time-passage actions 
only or discrete actions only. 

Proof. The context C" is built from C as follows: 

1. for each time-passage transition s — ► s' of C and each trajectory uj for s — ► s' , add an 
action starts and an action end w ; 

2. for each time-passage transition s — ► s' of C and each trajectory uj for s — ► s' , add a 
collection of new states {s w ,t | < t < d}, a transition s — > s^o, a transition s w ^ — " s' , 
and for each < t < t' < d, a transition s Wji — ► s w,t'] 

3. remove all the time-passage transitions leaving from states of C . 

Let V be a timed trace distribution of Mi||C that is not a timed trace distribution of M2IIC. 
Consider a probabilistic execution H\ of Mi||C such that t-tdistr(Hi) = V, and consider the 
scheduler that leads to Hi. Apply to Mi||C" the same scheduler with the following modification: 
whenever a time-passage transition s — ► s' is scheduled, choose a trajectory uj for s — > s' 
and schedule starts, followed by d, and followed by end w . Denote the resulting probabilistic 
execution by H[ and the resulting timed trace distribution by V. Then, 

V \ acts(Mi\\C) = V. (11.13) 

To prove (11.13) we prove first that tdistr(H[) \ acts(Mi\\C) = tdistr(Hi), and then we apply 
Propositions 11.3.6 and 11.3.7. To prove that tdistr(H[) \ acts(Mi\\C) = tdistr(Hi) we define 
a construction tclp to be applied to probabilistic executions of M 8 ||C" where each occurrence of 
a start action is followed eventually by the corresponding end action with probability 1. 

Let H' be a probabilistic execution of M 8 ||C" where each occurrence of a start action is 
followed eventually by the corresponding end action with probability 1, and denote tclp(H') by 
H . For each state q of H', let tclp(q) be obtained from q by replacing each state of the form s Wji 
with the state uj(t), by removing each occurrence of a start action together with its following 
state, and by removing each end action together with its following state. Then, 

states(H) = tclp(states(H')). (H-14) 

Let (q,V) be a restricted transition of H', and suppose that no start or end action occurs. Let 
fi' = {(a, tclp(q')) I (a,q') G fi}, and for each (a,q") G fi', let P'[(a,q")] = P[a X tclp^iq")], 
where te/p _1 (g) is the set of states q' of H' such that tclp(q') = q. Then the transition 
tclp((q,V)) is defined to be 

tclp((q,V)) = (tclp(q),V). (11.15) 
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For the transition relation of H , consider a state q of H , and let min(tclp (qj) be the set of 
minimal states of £c/p _1 (g) under prefix ordering. For each state q £ £c/p _1 (g), let 

pt'b-Hi) ± Pw[Cg\ ^ (11.16) 

The transition enabled from q in H is 

J2 pj' p ~ 1{q) PF'[acts(Mi\\C)]tclp(tr$' \ acts(Mi\\C)). (11.17) 

g'Gic/j)" 1 ^) 

The probabilistic execution H satisfies the following properties. 

a. if is a probabilistic execution of M 8 ||C. 

The fact that each state of H is reachable can be shown by a simple inductive argument; 
the fact that each state of if is a finite execution fragment of M 8 ||C follows from a simple 
analysis of the definition of tclp. 

From (11.17) it is enough to check that for each state q' of H' , the transition tclp(tr^, \ 
acts(Mi\\C)) is generated a combined transition of M 8 ||C. Since tr^, is a transition of 
H' , (tr^i \ acts(Mi\\C)) can be expressed as q' ~ tr, where tr is a combined transition of 
Mj-||C" and no start or end action occurs in tr. Let tr' be obtained by substituting each 
state of the form s Wji with uj(t) in tr. Then, tr' is a combined transition of M||C, and, 
from the definition of tclp, tclp(tr^, \ acts(Mi\\Cj) = tclp(q') ~ tr'. 

b. For each state q of H , 

P H [C q }= Y, Pw[C q *]. (11.18) 

q' (zTtiin(tclp~ (g)) 

This is shown by induction on the length of q. If q consists of a start state only, then 
the result is trivial. Otherwise, from the definition of the probability of a cone, Equa- 
tion (11.17), and a simple algebraic simplification, 

P H [C qas } = P H [C q ] | £ p?~ 1(9) Pf [a X tclp-\qas)] ) . (11.19) 




Observe that for each q' £ £c/p _1 (g) the set fl^, n ({a} X tclp -1 (qas)) contains only one 
element, say (a, q'as"), and thus Pfji[C q i]Pff [ax tclp~ (qas)] gives PH'[C q i a s"]- Moreover, 
observe that the states of min(tclp~ l (qas)) are the states of the form described in Equa- 
tion (11.19) (simple cases analysis). Thus, by applying induction to (11.19), using (11.16), 
simplifying algebraically, and using the observations above, 

P H [C qas }= J2 PH>[C q .]. (11.20) 

q' £min(tclp~ (qas)) 
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c. tdistr(H) = tdistr(H') \ acts(M t \\C). 

Let (3 be a finite trace of H or H'. Then {a £ 0#/ | (3 < trace(a) \ acts(Mi\\C)} can be 
expressed as a union of disjoint cones U ge eCg where 

= {q e states(H') \ trace(q) \ acts(M t \\C) = /3, lact(q) = lact{fi)}. (11.21) 

The set tclp(Q) is the set 

tclp(Q) = {q £ states(H) \ trace(q) = (3, lact(q) = lact(fi)}, (11.22) 

which is a characterization of {a £ S7# | (3 < trace(a)} as a union of disjoint cones. Ob- 
serve that min(tclp~ (tclp(Q)j) = 0. Moreover, for each q\ ^ q 2 of tclp(Q), tclp~ (q\) n 
tclp~ 1 (q 2 ) = 0. Thus, from (11.18), P HI [U ge @C g ] = PH[U q etclp(&)C g ]. This is enough to 
conclude. 

To complete the proof of (11.13) it is enough to observe that Hi = tclp(H[). Property (11.13) 
is then expressed by property (c). 

Suppose by contradiction that it is possible to obtain V from M2IIC". Consider the scheduler 
that leads to V in M2IIC", and let H' 2 be the corresponding probabilistic execution. Observe 
that, since the timed trace distribution of H' 2 is V , and since by construction in V each occur- 
rence of a start action is followed eventually by the corresponding end action with probability 
1, in H' 2 each occurrence of a start action is followed eventually by the corresponding end 
action with probability 1. Thus, tclp can be applied, and t-tdistr^tclp^H^)) = V, which is a 
contradiction. ■ 

Lemma 11.5.6 Let C be a distinguishing time- deterministic, cycle-free, separated context for 
two probabilistic timed automata Mi and M 2 that from any state enables either time-passage 
actions only or discrete actions only. Then there exists a distinguishing time-deterministic, 
cycle-free separated context C" for Mi and M 2 that from any state enables either time-passage 
actions only or discrete actions only, and such that the transition relation from any state 
enabling discrete actions is at most countably branching. C" is called a time-deterministic, 
countably-branching, cycle-free separated context. 

Proof. Let V a timed trace distribution of Mi||C that is not a timed trace distribution of 
M2HC. Consider one of the corresponding probabilistic executions H. Observe that H has at 
most countably many states that enable discrete actions, and that at each state of H there are 
at most countably many transitions of C that are scheduled. Thus, in total, only countably 
many discrete transitions of C are used to generate V. Then C" is C without the useless discrete 
transitions. ■ 

Lemma 11.5.7 Let C be a distinguishing time-deterministic, countably-branching, cycle-free 
separated context for two probabilistic timed automata Mi and M 2 . Then there exists a dis- 
tinguishing cycle-free separated context C" for Mi and M 2 that at each state enabling discrete 
actions either enables two deterministic transitions or a unique probabilistic transition with two 
possible outcomes. C" is called a time-deterministic, binary separated context. 
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Proof. The context C" is built from C in the same way as in the proof of Lemma 7.5.6. The 
constructions shr and shf work as well. The specific procedure is the same as the procedure 
followed in the proof of Lemma 11.5.3. ■ 

Lemma 11.5.8 Let C be a distinguishing time- deterministic, binary separated context for two 
probabilistic timed automata Mi and Mi . Then there exists a distinguishing time- deterministic, 
binary separated context C" for Mi and Mi where all the probabilistic transitions have a uniform 
distribution over two states. C" is called a time-deterministic, balanced separated context. 

Proof. The context C" is built from C in the same way as in the proof of Lemma 7.5.7. The 
specific procedure is the same as the procedure followed in the proof of Lemma 11.5.3. ■ 

Lemma 11.5.9 Let C be a distinguishing time-deterministic, balanced separated context for two 
probabilistic timed automata Mi and Mi . Then there exists a distinguishing time- deterministic, 
binary separated context C" for Mi and Mi with no internal actions and such that for each time 
t each discrete action appears exactly in one edge of the transition tree that leaves from a state 
whose time is t. C" is called a time-deterministic, total balanced separated context. 

Proof. The context C" is obtained from C by renaming all of its discrete actions so that for 
each time t each edge of the new transition relation leaving from a state whose current time is 
t has its own action. The proof of Lemma 7.5.8 applies. ■ 

Lemma 11.5.10 Let C be a distinguishing time-deterministic, total balanced separated context 
for two probabilistic timed automata Mi and Mi- Then there exists a distinguishing time- 
deterministic, total, cycle-free separated context C" for Mi and Mi that from every state en- 
ables one time-passage transition for each timed-action d, two deterministic transitions, and a 
probabilistic transition with a uniform distribution over two choices. C" is called a complete 
context. 

Proof. In this case it is enough to complete C by adding all the missing transitions and states. 
If V is a timed trace distribution of Mi||C that is not a timed trace distribution of M2IIC, then 
it is enough to use on Mi||C" the same scheduler that is used in Mi||C. In fact, since each new 
discrete transition of C" has a distinct action, none of the new discrete transitions of C" can be 
used in M2IIC" to generate V, and since each state of C" is uniquely determined by the timed 
trace of all the executions leading to that state, none of the new time-passage transitions can 
be scheduled (this would affect the resulting timed trace distribution). ■ 

Lemma 11.5.11 Let C be a distinguishing complete context for two probabilistic timed au- 
tomata Mi and Mi . Then the timed principal context is a distinguishing context for Mi and 
M 2 . 

Proof. The result is achieved in two steps. First the actions of C are renamed so that each 
state enables two deterministic transitions with actions left and right, a probabilistic transition 
with actions pleft and pright, and one transition for each time-passage action d. Call this 
context C\. Then, by observing that the state of C\ is uniquely determined by the timed trace 
of any timed execution leading to it, all the states of C\ are collapsed into a unique one. 

Thus, we need to show only that C\ is a distinguishing context. The proof of Lemma 7.5.10 
applies. ■ 
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Lemma 11.5.12 Let Cp be a distinguishing context for two probabilistic timed automata M\ 
and M 2 . Then the simple context C of Figure 7-6 augmented with a self-loop time-passage 
transition from state so for each time-passage action d, where start is an action that does not 
appear in Mi and M 2 , is a distinguishing context for M\ and M 2 . 

Proof. The proof of Lemma 7.5.11 applies. ■ 

Proof of Theorem 11.5.1. Let M x Q DCt M 2 . Then, from Lemma 11.5.12, Mi||C P C Dt 
M 2 \\C P . Conversely, let M^C'p Q Dt M 2 \\C P . Then, from Lemmas 11.5.3, 11.5.4, 11.5.5, 
11.5.6, 11.5.7, 11.5.8, 11.5.9, 11.5.10, and 11.5.11, M x \Z DCt M 2 . ■ 
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Chapter 12 

Hierarchical Verication Timed 
Simulations 

12.1 Introduction 

The simulation method extends to the timed framework almost directly. The main difference 
is that in a timed simulation that abstracts from internal computation we use moves (cf. Sec- 
tion 9.4) rather than weak combined transitions. The kind of results that we prove are a direct 
extension of similar results for the untimed model. In particular, probabilistic timed forward 
simulations are sound for the timed trace distribution precongruence. 

12.2 Probabilistic Timed Simulations 

We start directly with simulation relations that abstract from internal computation; the strong 
relations are essentially the same as for the untimed case. 

For convenience assume that Mi and Mi do not have common states. A probabilistic timed 
bisimulation between two simple probabilistic timed automata Mi and Mi is an equivalence 
relation 1Z over states(M\) U states^M?) such that 

1. each start state of Mi is related to at least one start state of M2, and vice versa; 

2. for each pair of states si 1Z S2 and each transition si — ► V\ of either Mi or M2, there 
exists a move .$2 ~~» V2 of either M\ or M2 such that V\ =-ji T>2- 

We write Mi ~p t M2 whenever ext(M\) = ext{M2) and there is a probabilistic timed bisimu- 
lation between Mi and M^- 

A probabilistic timed simulation between two simple probabilistic timed automata Mi and 
M2 is a relation 1ZC states(M\) X states(M2) such that 

1. each start state of Mi is related to at least one start state of M2; 

2. for each pair of states si 1Z S2 and each transition si — > V\ of Mi, there exists a move 
s 2 aUx S M2) V 2 of M 2 such that V x C K V 2 . 
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We write Mi Cp t M 2 whenever ext(M\) = e;zrf(M 2 ) and there is a probabilistic timed simulation 
from Mi to M 2 . We denote the kernel of probabilistic timed simulation by =pt- 

It is easy to check that ~p t is an equivalence relation, that Cp t is a preorder relation, and 
that both ~p t and Cp t are preserved by the parallel composition operator. It is also easy to 
verify that a weak probabilistic bisimulation is a probabilistic timed bisimulation and that a 
weak probabilistic simulation is a probabilistic timed bisimulation. 

12.3 Probabilistic Timed Forward Simulations 

A probabilistic timed forward simulation between two simple probabilistic timed automata 
Mi,M2 is a relation 1ZC states(M\) X Probs(states(M2)) such that 

1. each start state of Mi is related to at least one Dirac distribution over a start state of 

M 2 ; 

2. for each s 1Z V', if s — ► Vi, then 

(a) for each s' G 0' there exists a probability space V s > such that s' ~~» V s i, and 

(b) there exists a probability space V[ of Probs(Probs(states(M2))) satisfying V\ Ck V[, 

such that Zs'en* P'[s']V a > = Zven[ P[[P]V. 
Denote the existence of a probabilistic timed forward simulation from Mi to M 2 by Mi ^FSt M 2 . 
Proposition 12.3.1 ^FSt is preserved by the parallel composition operator. 

Proof. Let Mi ^FSt M 2 , and let 1Z be a probabilistic timed forward simulation from Mi to 
M 2 . Let 1Z' be a relation between states(M\) X states(M^) and Probs(states(M2) X states(M^)), 
defined as follows: 

Oi, s 3 ) W V iff V = V 2 ® V(s 3 ) for some V 2 such that s x U V 2 - 

The proof that 1Z' satisfies Condition 1 and that Condition 2 is satisfied for each discrete 
transition of Mi||M 2 is essentially the proof of Proposition 8.5.1. Thus we need to show only 
that Condition 2 is satisfied by time-passage transitions. 

Let (51,53) 1Z' V2 ® V(ss), and let (51,53) — ► (5^,53), where si — > s[, and 53 — ► 5 3 . 
From the definition of a probabilistic timed forward simulation, for each 5 £ S7 2 there exists 

a move 5 2 ~~» V s of M 2 , and there exists a probability space V'2 of Probs(Probs(states(M2))) , 
such that 

E P ^]Ps = £ I* 2 [P]V, (12.1) 



s£fi 2 ^£^2 



and 



X»(si) Qn V 2 . (12.2) 



Moreover, from the definition of a probabilistic timed automaton, there is a trajectory 0^3 for 

d , 
53 — ► 5 3 . 

For each 5 G i7 2 , let S be a generator for 5 ~~» V s . Define a new generator 0' s as follows: 
for each finite execution fragment a of M 2 ||M3 starting in (5, 53), 
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1. if O s (a\M2) = (s 1 ,V), where (s 1 ,V) = ^2iPi(s',ai,Vi), each (s',a,i,Vi) is a transition 
of M2, and a\Mz is consistent with U3, i.e., for each prefix a' of a, /sfafe(a') [M3 = 
tjj3(ltime(a')), then letting s" denote lstate(a\Ms), 

, s (a) = Y^ Pi ((s',4),a i ,V i ®Vl), 

i 

where V[ = T>(s'£) if a 8 - is a discrete action, and V[ = V(u3(ltime(a) + a 8 )) if a 8 - is a 
time-passage action. 

2. otherwise, 0£(a) = V(6). 

The move generated by each 0' s is (5,53) ~~» V s ® V(s' 3 ). In fact, an execution fragment a 
of M2HM3 is terminal for 0' s iff a\M2 is terminal for O s and /state(a[~M3) = s 3 , and thus 
0^; = J7 S x X , (sg). Moreover, for each a £ Oev, P« s = P a (^ • 

Denote V s ®V(s' 3 ) by Vr s S3 \. Then, for each (s, S3) G O2 ® ^(53), we have identified a move 
(s, S3) ~~» Vt s ^ ? \. These are the spaces of Condition 2. a in the definition of a probabilistic timed 
forward simulation. 

From this point the proof proceeds exactly in the same way as the proof of Proposition 8.5.1. 
No modification of the text is necessary. ■ 

12.4 The Execution Correspondence Theorem: Timed Ver- 
sion 

The execution correspondence theorem of Chapter 8 extends easily to the timed framework. In 
this section we define the notion of a timed execution correspondence structure, show the timed 
version of the execution correspondence theorem, and, as a consequence, show that probabilistic 
timed forward simulations are transitive. 

The timed execution correspondence theorem is stated in terms of the probabilistic execu- 
tions of a probabilistic timed automaton; however, it is easy to see that the same result can be 
extended to probabilistic timed executions: the execution correspondence theorem talks about 
countably many states of a probabilistic timed execution; all the other points can be described 
by arbitrary trajectories. 

12.4.1 Timed Execution Correspondence Structure 

The definition of a fringe for a probabilistic timed execution is the same as the definition of a 
fringe for a probabilistic execution. For the definition of fringe(H,i) the only difference is in 
the way the length of a state of H is measured, and thus the definition given for probabilistic 
automata is still valid. 

Let 1Z be a probabilistic timed forward simulation from Mi to M^- A timed execution corre- 
spondence structure via 1Z is a tuple (Hi, Hi, m, S), where Hi is a probabilistic execution of Mi, 
Hi is a probabilistic execution of Mi , m is a mapping from natural numbers to fringes of Mi , and 
S is a mapping from natural numbers to probability distributions of Probs(Probs(states(H2))), 
such that 
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1. For each i, m(i) < m(i + 1); 

2. For each state g 2 of H 2 , lim^^ Y, q en t \g 2 <g P ^ = P H[C q ]] 

3. Let qi 1Z (£l,J-,P) iff for each g£fi, t-trace(q) = t-trace(qi), and either 

(a) gi does not end in 8, each state of S7 does not end in 8, and Istate(qi) 1Z Istate(V), 
or 

(b) gi and each state of S7 end in 8 and lstate(8- strip(qi)) 1Z lstate(8- strip(V)) ■ 
Then, for each i > 0, m(i) = Y.ven s(l) p s(i)[P\P-, and fringe(H 1 ,i) Qn S(i). 

4. Let, for each i > 0, each gi £ fringe(Hi,i), and each g2 £ states^Hz), Wi(q\,q2) = 
J2v w i(Qi^T :> ) p [Q2\- If ^(gi,^^) = for each prefix or extension q' 2 of q 2 , then, for each 
extension q[ of gi such that q[ G fringe(Hi,i + 1), and each prefix or extension q' 2 of g2, 
W t+1 (q' 1 ,q' 2 ) = 0. 

12.4.2 The Main Theorem 

Theorem 12.4.1 Z-e£ Mi C^?5 M 2 via the probabilistic timed forward simulation 1Z, and let 
Hi be a probabilistic execution of Mi. Then there exists a probabilistic execution H 2 of M 2 , a 
mapping m from natural numbers to fringes of M 2 , and a mapping S from natural numbers to 
probability distributions of Probs(Probs(states(H 2 ))), such that (Hi, H 2 , m, S) is an execution 
correspondence structure via 1Z. 

Proof. The proof has exactly the same structure as the proof of Theorem 8.6.1. Note that the 
only difference between this theorem and Theorem 8.6.1 is in Condition 3, where we use timed 
traces rather than traces. ■ 

12.4.3 Transitivity of Probabilistic Timed Forward Simulations 

The timed execution correspondence theorem can be used to show that probabilistic timed 
forward simulations are transitive, i.e., if Mi ^FSt M 2 and M 2 ^FSt M%, then Mi ^FSt M%. 
The proof of this result follows the same lines as the corresponding proof in the untimed case 
(cf. Section 8.6.4), where combined transitions are replaced by moves and traces are replaced 
by timed traces. We leave the details of the proof to the reader. 

12.5 Soundness for Timed Trace Distributions 

As for the untimed model, the timed execution correspondence theorem can be used to show 
that probabilistic timed forward simulations are sound for the timed trace distribution precon- 
gruence. Since ^FSt is a precongruence, it is enough to show that ^FSt is sound for the timed 
trace distribution preorder. 

Proposition 12.5.1 If M x Q F St M 2 , then M x C Dt M 2 . 
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Proof. Let Mi ^FSt M2, and let Hi be a probabilistic execution of Mi that leads to a timed 
trace distribution T>\. From Lemma 12.4.1, there exists a probabilistic execution H2 of M2 
that corresponds to H\ via some mappings m, S. We show that H2 leads to a timed trace 
distribution T>2 that is equivalent to T>\. 

Consider a cone Cp of X>i . The cone Cp can be expressed as a union of cones of Vh x , and 
thus its measure can be expressed as 

.Urn J2 PhACiA- (12.3) 

Ql £fringe(Hi ,i)\f3<.t-trace(qi) 

Consider a cone Cp of T>2- The cone Cp can be expressed as a union of cones of Vh 2 i an( i thus 
its measure can be expressed as 

lim J2 Pm(i)l<l2]. (12.4) 

% — >oo — 

q 2 (z'in(i)\f3<t-trace(q2) 

The reason for Expression (12.4) is that at the limit each cone expressing the occurrence of (3 
is captured completely. 

Thus, it is sufficient to show that for each finite (3 and each i, 

E PnAC qi ]= E P ™«M- (12-5) 

Ql £fringe(Hi ,i)\f3<.t- trace (qi) q2{zm(i)\f3<.t-trace(q2) 

From this point the proof proceeds exactly as the proof of Proposition 8.7.1. ■ 
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Chapter 13 

Conclusion 

13.1 Have we Met the Challenge? 

We have developed a model for the description of randomized distributed real-time systems, and 
we have investigated how the new model can be used for the analysis of algorithms. The main 
idea behind the model is to extend labeled transition systems to account for randomization in 
such a way that probabilistic behavior and nondeterministic behavior are clearly distinct. 

We have shown how commonly used informal statements can be formulated in the new 
formalism, and we have shown how such statements can be proved to be correct in a formal 
and rigorous way. In particular, we have developed verification techniques that resemble the 
common ways in which randomized algorithms are analyzed. The main improvement is that 
now we have a collection of results that allow us to determine when a specific argument can be 
used safely. Furthermore, we have shown how to derive upper bounds to the complexity of a 
randomized distributed algorithm using an ordinary time complexity measure as well as more 
abstract complexity measures like "number of rounds in an asynchronous computation". 

Finally, we have extended several verification techniques that are commonly used within the 
labeled transition system model. We have extended the trace semantics of labeled transition 
systems and several of the existing simulation relations for labeled transition systems. In 
particular, all our preorder relations are compositional and the simulation relations are sound 
for the trace-based semantics. Although we have not presented any example of verification 
using simulations, except for two toy examples based on coin flips, we are confident that in the 
future the method based on simulations will become of practical relevance as it happened for 
ordinary automata. 

Therefore, we can claim that we have met the challenge given by randomization at least 
partially. Surely we understand much more of the problem than before. The fact that we have 
been able to prove new results about randomized algorithms is a positive sign. In particular, 
Aggarwal [Agg94] used successfully the technique presented in this thesis for the verification of 
the randomized self- stabilizing algorithm of Aggarwal and Kutten [AK93], which is not trivial 
at all; during the verification process Aggarwal discovered also a subtle bug in the original 
protocol. In the measure in which the power of a proof method is evaluated based on the bugs 
that such method helps to discover, our methodology has achieved something. Indeed we have 
discovered another bug on one existing algorithm, and the main issue is that we did not have 
to work much to discover such a bug; essentially it was sufficient to try to reformulate the proof 

263 



of correctness in our framework. 

13.2 The Challenge Continues 

Although we have improved considerably our understanding of randomization in distributed 
computation, what we have discovered looks like the tip of the iceberg. We have addressed 
several problems, and in solving them we have addressed more the basic methodology rather 
than an extensive analysis of all the possible solutions. Therefore, there are several directions for 
further research that can be pursued. Here we suggest some of the most important directions. 

13.2.1 Discrete versus Continuous Distributions 

Throughout this thesis we have assumed that the probability distributions associated with the 
transitions of a probabilistic automaton are discrete. Although such assumption is sufficiently 
general for the study of several randomized algorithms, several other real-time systems are better 
described by using continuous distributions. Examples involve algorithms for transmission of 
data along a common wire, scheduling algorithms for massively parallel machines, and queuing 
systems. Moreover, continuous distributions would be more suitable for the study of randomized 
hybrid systems. 

The extension of the theory to continuous distributions involves nontrivial measure theoret- 
ical problems. In particular it is not the case any more that any union of cones is measurable; 
thus, not even the event that expresses the occurrence of an action or the reachability of a 
state is measurable in general. The events with probability need a more careful treatment 
within the model with continuous distributions. It is likely that some restrictions must be 
imposed to the model to ensure that some minimal set of events is measurable. Examples of 
restricted models with continuous distributions are the automata of Alur, Courcuobetis and 
Dill [ACD91a, ACD91b], where the time that elapses between two transitions is governed by 
an exponential distribution or by a distribution which is non zero in a finite collection of closed 
intervals, and the models of [GHR93, Hil93, BDG94], where the time between the occurrence 
of two actions is assumed to be distributed exponentially. Exponential distributions occur in 
several real systems and are easy to model due to their memoryless structure. However, other 
distributions should be studied. 

13.2.2 Simplified Models 

Within the context of ordinary automata Lynch and Tuttle [LT87] have developed a model of 
I/O automata. The model enforces a distinction between Input actions and Output actions 
within an automaton, and requires that input actions are enabled from every state. Further- 
more, in a parallel composition context each action is required to be the output or internal 
action of at most one process, i.e., each action is under the control of at most one process. 
Based on the Input/Output distinction Lynch and Tuttle can introduce fairness in the model 
in a natural way, and in particular they can use the trace semantics as a meaningful notion of 
implementation. In general the trace semantics is not meaningful as a notion of implementation 
since, for example, it is not sensitive to deadlock. The advantage of the use of traces is that 
traces are easy to deal with. 
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Figure 13-1: Synchronization for probabilistic I/O automata. 

For this reason, it makes sense to study a theory of probabilistic I/O automata as an 
extension of the model of [LT87] and as a restriction of our model. An interesting point of a 
model with I/O distinction is that it is possible to relax the requirement that all the transitions 
of a probabilistic I/O automaton are simple. In particular, only the transitions with input 
actions need to be simple, while all the others can be general. The parallel composition can be 
defined easily since a non-simple transition synchronizes only with simple transitions. Figure 13- 
1 gives an example of synchronization between a transition with three output actions a, 6, c and 
two transitions of an I/O automaton with just two input actions a, b. A similar observation 
was made also by Wu, Stark and Smolka in [WSS94]. 

A restricted timed model with I/O distinction is introduced by Merrit, Modugno and Tuttle 
[MMT91]. In particular timing constraints can be described only by giving upper and lower 
bounds to the time it takes for a process to perform the next transition whenever it is ready 
to do so. MMT automata turned out to be sufficient for the modeling of several distributed 
systems, and in particular, due to their simple structure, made the analysis simpler than by 
using the full automaton model. Once again, a study of the probabilistic version of the MMT 
model would be useful. The proofs that we have illustrated in Chapter 12 could be carried out 
in the probabilistic MMT model as well. 

Finally, the analysis of a system can be simplified by studying time- deterministic probabilis- 
tic timed automata, i.e., probabilistic timed automata such that from each state s and each time 
d there is at most one state reachable from s in time d. In fact, if a system is time-deterministic, 
then the end points of a time-passage transition determine completely the trajectory that is 
spanned. Therefore, trajectories could be removed also from the direct analysis of randomized 
timed algorithms. It turns out that most of the times an algorithm can be described as a 
time-deterministic probabilistic automaton. Probabilistic MMT automata are an example of 
time-deterministic probabilistic automata. 

13.2.3 Beyond Simple Probabilistic Automata 

The study of parallel composition and of the simulation relations of this thesis is done within 
the context of simple probabilistic automata. The main problem is that we did not find any 
reasonable definition of parallel composition for general probabilistic automata that is consistent 
with our synchronization style. We have just observed that in the presence of an Input/Output 
distinction it is possible to relax the simplicity condition and yet obtain a meaningful notion 
of parallel composition. It would be interesting to investigate other mechanisms that give a 
meaning to general probabilistic automata and yet work as we expect in the simple case. 
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13.2.4 Completeness of the Simulation Method 

We have provided several simulation and bisimulation relations for probabilistic automata and 
probabilistic timed automata, and we have shown that they are sound for the trace distribution 
precongruence and the timed trace distribution precongruence, respectively. However, we have 
not shown any completeness result for probabilistic forward simulations and probabilistic for- 
ward timed simulations. In [LV93a, LV95] it is shown that forward simulations together with 
another kind of simulations called backward simulations are sound and complete for the trace 
preorder. Are probabilistic forward simulations complete for the trace distribution preorder? 
If not, is there an equivalent of backward simulations that can lead to completeness? 

13.2.5 Testing Probabilistic Automata 

We have presented the trace distribution semantics as an example of a semantics based on 
abstract observations. Another widely known semantics for ordinary automata is the failure 
semantics of Brookes, Hoare and Roscoe [BHR84], which in turn is connected to the testing 
preorders of De Nicola and Hennessy [DH84]. Similarly to the trace distribution semantics, 
it should be possible to extend the failure semantics to the probabilistic framework and find 
a sufficiently powerful context to distinguish probabilistic automata that are not in the corre- 
sponding precongruence relation. Possibly, a related theory of testing in the style of [DH84] 
should be defined. It is very likely that the new testing preorders will be similar to those 
of Yi and Larsen [YL92]. Other theories of testing for probabilistic automata are studied in 
[Chr90b, Chr90a, CSZ92, YCDS94] and are explained in Section 2.2. 

13.2.6 Liveness in Probabilistic Automata 

In the extension of the notion of an execution of an automaton we have obtained a parallelism 
between the theory of ordinary automata and the theory of probabilistic automata. In this 
parallelism also the notion of liveness has found its place, although we have not addressed the 
issue in this thesis. In ongoing research we have given a simple definition of a live probabilistic 
automaton as a pair (M, L) where L is an arbitrary subset of the probabilistic executions of M, 
and we have shown that the live trace distribution precongruence can be defined easily and can 
be characterized by a live principal context, which is essentially the principal context paired 
with the set of its probabilistic executions. However, lot of work remains to be done within the 
theory of liveness. 

First of all it would be useful to study how the definition of safety and liveness properties 
of Alpern and Schneider [AS85] extends to the probabilistic framework and what consequences 
such extension has. Furthermore, the use of the live trace preorder within ordinary automata 
makes sense as a notion of implementation in the presence of I/O distinction and of a property 
called receptiveness or environment-freedom [Dil88, AL93, GSSL94]. It would be useful to 
study the theory of receptiveness of [Dil88, AL93] and of environment-freedom of [GSSL94] 
in the context of randomization. In this case, differently from [GSSL94], the environment is 
expressed by a function rather than by a sequence of actions. However, non-trivial problems 
arise in imposing restrictions to the behavior of the environment. 
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13.2.7 Temporal Logics for Probabilistic Systems 

In the chapters on direct analysis we have identified a collection of probabilistic statements 
that are useful for the analysis of algorithms. However, there are several other statements that 
can be of interest. It would be desirable to find a probabilistic temporal logic that expresses 
as many properties as possible. The probabilistic modal logic of [LS89] is a direct extension of 
the modal logic of Hennessy and Milner [HM85] for reactive processes, but it is not sufficiently 
powerful to deal with nondeterminism; similarly, the extended probabilistic logic of [LS92] is not 
sufficiently powerful. The Probabilistic Computation Tree Logic of [HJ89, Han94] captures more 
the consequences of the interplay between probability and nondeterminism; in [SL94] PCTL is 
generalized also to probabilistic systems with internal actions (WPCTL). However, there are 
still properties that are useful and do not seem to be expressible in WPCTL. Specifically, we 
do not know how to express a property of the kind "after something has happened, no matter 
where I am, something else will happen with probability at least p". Is there something missing 
in WPCTL? What would be a more appropriate temporal logic? 

Another issue is the relationship between the simulation method and temporal logic. That 
is, if a probabilistic automaton implements another probabilistic automaton according to some 
implementation relation (e.g., trace distribution precongruence, probabilistic simulation, proba- 
bilistic forward simulation, etc.), what can we say about the implementation? What properties 
of the specification are satisfied by the implementation? More generally, given a probabilis- 
tic temporal logic and a preorder relation, what fragment of the logic is preserved by the 
preorder relation? Somehow it is implicit that whenever we use some preorder relation as a 
notion of implementation we are interested only in the properties that are preserved by such 
relation; however, we need to know what are those properties. In [SL95] we have stated that 
weak probabilistic simulation preserve a large fragment of WPCTL and that weak probabilistic 
bisimulations preserve WPCTL. The results of [SL95] can be proved easily given the results of 
this thesis. However, more work in this direction is necessary. In particular, some completeness 
results would be useful. 

13.2.8 More Algorithms to Verify 

In this thesis we have illustrated our direct verification technique by proving the correctness 
of the randomized dining philosophers algorithm of Lehmann and Rabin [LR81] and of the 
randomized agreement protocol of Ben-Or [B083]. In [Agg94] Aggarwaluses our model to verify 
the correctness of the self- stabilizing minimum weight spanning tree randomized algorithm of 
Aggarwal and Kutten [AK93]. However, the technique should be tested against many other 
algorithms. We are currently investigating the agreement protocol of Aspnes and Herlihy [AH90] 
and the randomized mutual exclusion algorithm of Pnueli and Zuck [PZ86]. Based on the little 
experience that we have gained, we can say that the model provides us with a systematic way 
of analyzing those algorithms, and in particular it provides us with a simple methodology to 
identify the critical points of an algorithm. 

It is very likely that new coin lemmas need to be developed together with other techniques 
for the actual computation of the probability of an event. A technique that needs further 
development is the partition technique of Section 6.7. The analysis of other algorithms should 
make clear what other techniques are necessary. Also, playing with the toy resource allocation 
protocol of Chapter 5 can be very instructive. Although the protocol is simple, its analysis 
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highlights several of the issues that arise in randomized distributed computation. 

It is also plausible, as it happened for non-probabilistic distributed algorithms, that some 
complex protocols can be verified more easily by using the simulation method. Finding those 
algorithms would be an optimal way to test the hierarchical verification method and possibly 
to improve it. 

13.2.9 Automatic Verification of Randomized Systems 

Formal verification usually involves two levels of analysis. First, an algorithm is analyzed at 
a high level by using the intuition that designers have of their own algorithm; then, a more 
detailed verification of the high level claims is carried out in order to guarantee correctness. 
The low level analysis is very tedious and involves checking a whole lot of uninteresting details. 
On the other hand, several times the low level analysis is the only way to discover flaws in the 
intuitions about an algorithm. 

Fortunately, the low level analysis is amenable to automatic verification, although the re- 
search in this area is still in progress. Model checking [EC82, CES83] is certainly a useful 
technique; in [SGG + 93] it is shown how a theorem prover can be used to help in the verification 
of a protocol using simulations; in [PS95] we have investigated how a randomized algorithm 
can be verified mechanically once the high level proof is formulated. However, there is still a 
lot of work that needs to be done. It would be interesting to study how model checking and 
theorem proving could be integrated to automatize part of the verification of an algorithm. 

13.3 The Conclusion's Conclusion 

To say what we have done in one sentence, we have provided a new way of reasoning about 
randomized systems that integrates both the theoretical aspects of modeling and the basic 
requirements for usage in practice. From the modeling point of view we have distinguished be- 
tween nondeterminism and probability explicitly and we have extended the main semantics that 
are available within the labeled transition systems model; from the point of view of verification 
we have formalized some of the common informal arguments about randomized algorithms and 
we have provided guidelines to determine whether an argument can be used safely. Further- 
more, we have provided a systematic way to analyze the complexity of randomized algorithms. 
All our results are compatible with previous work. 

As we have seen in the previous section, there are still many open problems in this area. 
Here we hope to have stimulated the curiosity of the reader to go much further. Needless to 
say that for us (me) working on this project was a continuous discovery. 
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